File size: 1,918 Bytes
2ee2cb8 447d8e7 34e25e0 1c63cc7 34e25e0 447d8e7 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 34e25e0 1c63cc7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
---
library_name: transformers
license: mit
---
# Arcanum-12b 🧙♂️
Arcanum-12b is a merged large language model created by combining TheDrummer/Rocinante-12B-v1.1 and MarinaraSpaghetti/NemoMix-Unleashed-12B using a novel merging technique.
## Model Details 📊
- **Developed by:** Xclbr7
- **Model type:** Causal Language Model
- **Language(s):** English (primarily), may support other languages
- **License:** [Specify the license]
- **Repository:** https://huggingface.co/Xclbr7/Arcanum-12b
## Model Architecture 🏗️
- **Base model:** MarinaraSpaghetti/NemoMix-Unleashed-12B
- **Parameter count:** ~12 billion
- **Context length:** [Specify the context length]
- **Vocabulary size:** [Specify the vocabulary size]
- **Architecture specifics:** Transformer-based language model
## Training & Merging 🔄
Arcanum-12b was created by merging two existing 12B models:
1. TheDrummer/Rocinante-12B-v1.1
- Density parameters: [1, 0.8, 0.6]
- Weight: 0.7
2. MarinaraSpaghetti/NemoMix-Unleashed-12B
- Density parameters: [0.5, 0.7, 0.9]
- Weight: 0.8
**Merging method:** Ties
**Additional parameters:**
- Normalization: True
- Int8 mask: True
- Data type: float16
## Intended Use 🎯
Conversation with different personas.
## Performance and Limitations ⚖️
Not tested yet.
## Ethical Considerations 🤔
As a merged model based on existing language models, Arcanum-12b may inherit biases and limitations from its parent models. Users should be aware of potential biases in generated content and use the model responsibly.
## Acknowledgments 🙏
We acknowledge the contributions of the original model creators:
- TheDrummer for Rocinante-12B-v1.1
- MarinaraSpaghetti for NemoMix-Unleashed-12B
Their work formed the foundation for Arcanum-12b.
|