---
language:
- en
pipeline_tag: text-generation
tags:
- not-for-all-audiences
---
## Notes
There is no template, just BOS+text

It can also start from nothing

Temperature, repetition penalty, etc should all be left as defaults

It will not go lewd immediately, it will try to form a coherent story

It's best to generate 1~3 paragraphs at a time, it loses coherence if you try to make it generate the full context all at once



## LLaMA-3-8B base
RoPEd to 16k context

| Name                                  | Quant  | Size | VRAM (With FA) | VRAM (No FA) |
|---------------------------------------|--------|------|----------------|--------------|
| llama-3-8b-lewd-stories-v6-16k.F16    | F16    | 14.9 | 16.6           | 17.4         |
| llama-3-8b-lewd-stories-v6-16k.Q8_0   | Q8_0   | 8.0  | 10.1           | 10.5         |
| llama-3-8b-lewd-stories-v6-16k.Q6_K   | Q6_K   | 6.1  | 8.4            | 9.2          |
| llama-3-8b-lewd-stories-v6-16k.Q5_K_M | Q5_K_M | 5.3  | 7.6            | 8.1          |
| llama-3-8b-lewd-stories-v6-16k.Q4_K_M | Q4_K_M | 4.6  | 6.9            | 7.8          |

## Yi-1.5-9B-32K

Native 32k context

| Name                       | Quant  | Size | VRAM (With FA) | VRAM (No FA) |
|----------------------------|--------|------|----------------|--------------|
| yi-lewd-stories-32k.F16    | F16    | 16.4 |                |              |
| yi-lewd-stories-32k.Q8_0   | Q8_0   | 8.7  |                |              |
| yi-lewd-stories-32k.Q6_K   | Q6_K   | 6.7  |                |              |
| yi-lewd-stories-32k.Q5_K_M | Q5_K_M | 5.8  |                |              |
| yi-lewd-stories-32k.Q4_K_M | Q4_K_M | 5.0  |                |              |



## Mistral-7B-v0.3

Native 32k context

| Name                            | Quant  | Size | VRAM (With FA) | VRAM (No FA) |
|---------------------------------|--------|------|----------------|--------------|
| mistral-lewd-stories-32k.F16    | F16    | 13.5 |                |              |
| mistral-lewd-stories-32k.Q8_0   | Q8_0   | 7.2  |                |              |
| mistral-lewd-stories-32k.Q6_K   | Q6_K   | 5.5  |                |              |
| mistral-lewd-stories-32k.Q5_K_M | Q5_K_M | 4.8  |                |              |
| mistral-lewd-stories-32k.Q4_K_M | Q4_K_M | 4.0  |                |              |