File size: 828 Bytes
962de4a 9435b22 962de4a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
---
license: cc-by-nc-4.0
tags:
- not-for-all-audiences
- nsfw
---
<!-- description start -->
Exllamav2 4.65bpw quantization of CausalLM-RP-34B from [NeverSleep](https://huggingface.co/NeverSleep/CausalLM-RP-34B), quantized with default calibration dataset.
> [!IMPORTANT]
>Fits in 24GB VRAM with 32k+ context. Make sure to enable 4-bit cache option or you'll run into OOM errors.
---
## Original Card
## Description
This repo contains fp16 files of CausalLM-RP-34B, a finetuned model of the CausalLM-34B Beta on multiple RP datasets.
<!-- description end -->
<!-- description start -->
## Model used
- [CausalLM/34b-beta](https://huggingface.co/CausalLM/34b-beta)
### Prompt template ChatML
```
<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
{output}<|im_end|>
``` |