--- base_model: - nothingiisreal/L3.1-8B-Celeste-V1.5 - Sao10K/Llama-3.1-8B-Stheno-v3.4 - Sao10K/L3.1-8B-Niitama-v1.1 - arcee-ai/Llama-3.1-SuperNova-Lite - akjindal53244/Llama-3.1-Storm-8B - arcee-ai/Llama-Spark - grimjim/Llama-3-Instruct-abliteration-LoRA-8B - crestf411/sunfall-peft - v000000/L3.1-Celestial-Stone-2x8B library_name: transformers tags: - merge - llama - mixtral - dpo --- [![QuantFactory Banner](https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ)](https://hf.co/QuantFactory) # QuantFactory/L3.1-Celestial-Stone-2x8B-DPO-GGUF This is quantized version of [v000000/L3.1-Celestial-Stone-2x8B-DPO](https://huggingface.co/v000000/L3.1-Celestial-Stone-2x8B-DPO) created using llama.cpp # Original Model Card > [!WARNING] > **Sampler:**
> Likes a low temperature due to the MoE architecture. I use 0.3 personally. # Llama-3.1-Celestial-Stone-2x8B-DPO (BF16) * *DPO Trained, Mixture of Experts (14B).* ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64f74b6e6389380c77562762/lyRa7z5maTqAaa43sxC2J.png) * Direct Preference Optimization run ----> [Q6_K](https://huggingface.co/v000000/L3.1-Celestial-Stone-2x8B-DPO-Q6_K-GGUF) --------------------------------------------------------------------------------- [L3.1-Celestial-Stone-2x8B](https://huggingface.co/v000000/L3.1-Celestial-Stone-2x8B) Finetuned on Nvidia A100. 0.5 Epoch completed of dataset [jondurbin/gutenberg-dpo-v0.1](https://huggingface.co/datasets/jondurbin/gutenberg-dpo-v0.1) with learning_rate=8e-6 Result seems pretty good. More compliant and verbose, less sloppy and safety aligned. ------------------------------------------------------------------------------ *The first expert* is Instruct 405B distillation/RP vector merge (Supernova-Lite, Niitama1.1, Storm) *The second expert* is ERP/Reddit data merge (Celeste1.5, Stheno3.4, Storm) ------------------------------------------------------------------------------- *The base model* is Sao10k/L3.1-Stheno-3.4 with the Sunfall LoRa 0.6.1 to make it understand SillyTavern prompts and storywriting better. ------------------------------------------------------------------------------- *Resultant merge finetuned* on [jondurbin/gutenberg-dpo-v0.1](https://huggingface.co/datasets/jondurbin/gutenberg-dpo-v0.1). # Prompt Template: ```bash <|begin_of_text|><|start_header_id|>system<|end_header_id|> {system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|> {input}<|eot_id|><|start_header_id|>assistant<|end_header_id|> {output}<|eot_id|> ```