|
--- |
|
tags: |
|
- merge |
|
- mergekit |
|
- lazymergekit |
|
- not-for-all-audiences |
|
- nsfw |
|
- rp |
|
- roleplay |
|
- role-play |
|
license: llama3 |
|
language: |
|
- en |
|
library_name: transformers |
|
pipeline_tag: text-generation |
|
base_model: |
|
- Sao10K/L3-8B-Stheno-v3.2 |
|
- Hastagaras/Jamet-8B-L3-MK.V-Blackroot |
|
- grimjim/Llama-3-Oasis-v1-OAS-8B |
|
- Casual-Autopsy/SOVL-MopeyMule-8B |
|
- Casual-Autopsy/MopeyMule-Blackroot-8B |
|
- ResplendentAI/Theory_of_Mind_Llama3 |
|
- ResplendentAI/RP_Format_QuoteAsterisk_Llama3 |
|
- ResplendentAI/Smarts_Llama3 |
|
- Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B |
|
- Casual-Autopsy/Llama-3-Mopeyfied-Psychology-8B |
|
- Hastagaras/Halu-8B-Llama3-Blackroot |
|
model-index: |
|
- name: L3-Umbral-Mind-RP-v1.0-8B |
|
results: |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: Unruly |
|
type: Unruly |
|
metrics: |
|
- name: accuracy |
|
type: acc |
|
value: 54.6 |
|
source: |
|
name: UGI Leaderboard |
|
url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: Internet |
|
type: Internet |
|
metrics: |
|
- name: accuracy |
|
type: acc |
|
value: 31.4 |
|
source: |
|
name: UGI Leaderboard |
|
url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: CrimeStats |
|
type: CrimeStats |
|
metrics: |
|
- name: accuracy |
|
type: acc |
|
value: 45 |
|
source: |
|
name: UGI Leaderboard |
|
url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: Stories/Jokes |
|
type: Stories/Jokes |
|
metrics: |
|
- name: accuracy |
|
type: acc |
|
value: 56.8 |
|
source: |
|
name: UGI Leaderboard |
|
url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: PolContro |
|
type: PolContro |
|
metrics: |
|
- name: accuracy |
|
type: acc |
|
value: 66.7 |
|
source: |
|
name: UGI Leaderboard |
|
url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: W/10 |
|
type: W/10 |
|
metrics: |
|
- name: willingness to answer |
|
type: willingness to answer |
|
value: 8 |
|
source: |
|
name: UGI Leaderboard |
|
url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard |
|
--- |
|
|
|
<img src="https://huggingface.co/Casual-Autopsy/L3-Umbral-Mind-RP-v3-8B/resolve/main/63073798_p0_master1200.jpg" style="display: block; margin: auto;"> |
|
Image by ろ47 |
|
|
|
**Highest ranked 8B model on the [UGI Leaderboard](https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard) as of writing this!** |
|
|
|
# Merge |
|
|
|
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). |
|
|
|
## Merge Details |
|
|
|
The goal of this merge was to make an RP model better suited for role-plays with heavy themes such as but not limited to: |
|
- Mental illness |
|
- Self-harm |
|
- Trauma |
|
- Suicide |
|
|
|
I hated how RP models tended to be overly positive and hopeful with role-plays involving such themes, |
|
but thanks to [failspy/Llama-3-8B-Instruct-MopeyMule](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule) this problem has been lessened considerably. |
|
|
|
If you're an enjoyer of savior/reverse savior type role-plays like myself, then this model is for you. |
|
|
|
### Usage Info |
|
|
|
This model is meant to be used with asterisks/quotes RPing formats, any other format that isn't asterisks/quotes is likely to cause issues |
|
|
|
### Quants |
|
|
|
- [Static GGUF quants](https://huggingface.co/mradermacher/L3-Umbral-Mind-RP-v1.0-8B-GGUF) by mradermacher |
|
- [Imatrix GGUF quants](https://huggingface.co/mradermacher/L3-Umbral-Mind-RP-v1.0-8B-i1-GGUF) by mradermacher |
|
- [Imatrix GGUF quants](https://huggingface.co/Lewdiculous/L3-Umbral-Mind-RP-v1.0-8B-GGUF-IQ-Imatrix) by Lewdiculous |
|
- exl2's by riveRiPH: |
|
- [L3-Umbral-Mind-RP-v1.0-8B-8bpw-h8-exl2](https://huggingface.co/riveRiPH/L3-Umbral-Mind-RP-v1.0-8B-8bpw-h8-exl2) |
|
- [L3-Umbral-Mind-RP-v1.0-8B-6.3bpw-h8-exl2](https://huggingface.co/riveRiPH/L3-Umbral-Mind-RP-v1.0-8B-6.3bpw-h8-exl2) |
|
- [L3-Umbral-Mind-RP-v1.0-8B-5.3bpw-h6-exl2](https://huggingface.co/riveRiPH/L3-Umbral-Mind-RP-v1.0-8B-5.3bpw-h6-exl2) |
|
|
|
### Merge Method |
|
|
|
This model was merged using several Task Arithmetic merges and then tied together with a Model Stock merge, followed by another Task Arithmetic merge with a model containing psychology data. |
|
|
|
### Models Merged |
|
|
|
The following models were included in the merge: |
|
* [Sao10K/L3-8B-Stheno-v3.2](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2) |
|
* [Hastagaras/Halu-8B-Llama3-Blackroot](Hastagaras/Halu-8B-Llama3-Blackroot) |
|
* [Casual-Autopsy/Llama-3-Mopeyfied-Psychology-8B](https://huggingface.co/Casual-Autopsy/Llama-3-Mopeyfied-Psychology-8B) |
|
* [Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B](https://huggingface.co/Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B) |
|
* Casual-Autopsy/Umbral-v3-1 + [ResplendentAI/Theory_of_Mind_Llama3](https://huggingface.co/ResplendentAI/Theory_of_Mind_Llama3) |
|
* [Sao10K/L3-8B-Stheno-v3.2](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2) |
|
* [Casual-Autopsy/SOVL-MopeyMule-8B](https://huggingface.co/Casual-Autopsy/SOVL-MopeyMule-8B) |
|
* [Casual-Autopsy/MopeyMule-Blackroot-8B](https://huggingface.co/Casual-Autopsy/MopeyMule-Blackroot-8B) |
|
|
|
* Casual-Autopsy/Umbral-v3-2 + [ResplendentAI/Smarts_Llama3](https://huggingface.co/ResplendentAI/Smarts_Llama3) |
|
* [Hastagaras/Jamet-8B-L3-MK.V-Blackroot](https://huggingface.co/Hastagaras/Jamet-8B-L3-MK.V-Blackroot) |
|
* [Casual-Autopsy/SOVL-MopeyMule-8B](https://huggingface.co/Casual-Autopsy/SOVL-MopeyMule-8B) |
|
* [Casual-Autopsy/MopeyMule-Blackroot-8B](https://huggingface.co/Casual-Autopsy/MopeyMule-Blackroot-8B) |
|
|
|
* Casual-Autopsy/Umbral-v3-3 + [ResplendentAI/RP_Format_QuoteAsterisk_Llama3](https://huggingface.co/ResplendentAI/RP_Format_QuoteAsterisk_Llama3) |
|
* [grimjim/Llama-3-Oasis-v1-OAS-8B](https://huggingface.co/grimjim/Llama-3-Oasis-v1-OAS-8B) |
|
* [Casual-Autopsy/SOVL-MopeyMule-8B](https://huggingface.co/Casual-Autopsy/SOVL-MopeyMule-8B) |
|
* [Casual-Autopsy/MopeyMule-Blackroot-8B](https://huggingface.co/Casual-Autopsy/MopeyMule-Blackroot-8B) |
|
|
|
## Secret Sauce |
|
|
|
The following YAML configurations were used to produce this model: |
|
|
|
### Umbral-v3-1 |
|
|
|
```yaml |
|
slices: |
|
- sources: |
|
- model: Sao10K/L3-8B-Stheno-v3.2 |
|
layer_range: [0, 32] |
|
parameters: |
|
weight: 0.65 |
|
- model: Casual-Autopsy/SOVL-MopeyMule-8B |
|
layer_range: [0, 32] |
|
parameters: |
|
weight: 0.25 |
|
- model: Casual-Autopsy/MopeyMule-Blackroot-8B |
|
layer_range: [0, 32] |
|
parameters: |
|
weight: 0.1 |
|
merge_method: task_arithmetic |
|
base_model: Sao10K/L3-8B-Stheno-v3.2 |
|
normalize: False |
|
dtype: bfloat16 |
|
``` |
|
|
|
### Umbral-v3-2 |
|
|
|
```yaml |
|
slices: |
|
- sources: |
|
- model: Hastagaras/Jamet-8B-L3-MK.V-Blackroot |
|
layer_range: [0, 32] |
|
parameters: |
|
weight: 0.75 |
|
- model: Casual-Autopsy/SOVL-MopeyMule-8B |
|
layer_range: [0, 32] |
|
parameters: |
|
weight: 0.15 |
|
- model: Casual-Autopsy/MopeyMule-Blackroot-8B |
|
layer_range: [0, 32] |
|
parameters: |
|
weight: 0.1 |
|
merge_method: task_arithmetic |
|
base_model: Hastagaras/Jamet-8B-L3-MK.V-Blackroot |
|
normalize: False |
|
dtype: bfloat16 |
|
``` |
|
|
|
### Umbral-v3-3 |
|
|
|
```yaml |
|
slices: |
|
- sources: |
|
- model: grimjim/Llama-3-Oasis-v1-OAS-8B |
|
layer_range: [0, 32] |
|
parameters: |
|
weight: 0.55 |
|
- model: Casual-Autopsy/SOVL-MopeyMule-8B |
|
layer_range: [0, 32] |
|
parameters: |
|
weight: 0.35 |
|
- model: Casual-Autopsy/MopeyMule-Blackroot-8B |
|
layer_range: [0, 32] |
|
parameters: |
|
weight: 0.1 |
|
merge_method: task_arithmetic |
|
base_model: grimjim/Llama-3-Oasis-v1-OAS-8B |
|
normalize: False |
|
dtype: bfloat16 |
|
``` |
|
|
|
### L3-Umbral-Mind-RP-v0.3-8B |
|
|
|
```yaml |
|
models: |
|
- model: Casual-Autopsy/Umbral-v3-1+ResplendentAI/Theory_of_Mind_Llama3 |
|
- model: Casual-Autopsy/Umbral-v3-2+ResplendentAI/Smarts_Llama3 |
|
- model: Casual-Autopsy/Umbral-v3-3+ResplendentAI/RP_Format_QuoteAsterisk_Llama3 |
|
merge_method: model_stock |
|
base_model: Casual-Autopsy/Umbral-v3-1 |
|
dtype: bfloat16 |
|
``` |
|
|
|
### L3-Umbral-Mind-RP-v1.0-8B |
|
|
|
```yaml |
|
slices: |
|
- sources: |
|
- model: Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B |
|
layer_range: [0, 32] |
|
- model: Casual-Autopsy/Llama-3-Mopeyfied-Psychology-8B |
|
layer_range: [0, 32] |
|
parameters: |
|
weight: 0.14 |
|
- model: Sao10K/L3-8B-Stheno-v3.2 |
|
layer_range: [0, 32] |
|
parameters: |
|
weight: 0.03 |
|
- model: Hastagaras/Halu-8B-Llama3-Blackroot |
|
layer_range: [0, 32] |
|
parameters: |
|
weight: 0.03 |
|
merge_method: task_arithmetic |
|
base_model: Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B |
|
dtype: bfloat16 |
|
``` |
