File size: 2,760 Bytes
1a32e7a d297c5f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
---
base_model: nintwentydo/Razorback-12B-v0.2
base_model_relation: quantized
library_name: transformers
tags:
- mergekit
- merge
- multimodal
- mistral
- pixtral
language:
- en
- fr
- de
- es
- it
- pt
- ru
- zh
- ja
license: other
pipeline_tag: image-text-to-text
---
# Razorback 12B v0.2 ExLlamaV2 5.0bpw Quant
#### UnslopNemo with Vision!
<img src="https://huggingface.co/nintwentydo/Razorback-12B-v0.1/resolve/main/razorback.jpg" style="width: 100%; max-width:700px"></img>
A more robust attempt at merging TheDrummer's UnslopNemo v3 into Pixtral 12B.
Has been really stable in my testing so far. Needs more testing to see what samplers it does/doesn't like.
Seems to be the best of both worlds - less sloppy, more engaging content and decent intelligence / visual understanding.
## Merging Approach
First, I loaded up Pixtral 12B Base and Mistral Nemo Base to compare their parameter differences.
Looking at the L2 norm / relative difference values I was able to isolate which parts of Pixtral 12B are a significant deviation from Mistral Nemo.
Because while the language model architecture is the same between the two, a lot of vision understanding has been trained into Pixtral's language model and can break very easily.
Then I calculated merging weights for each parameter using an exponential falloff. The smaller the difference, the higher the weight.
Applied this recipe to Pixtral Instruct (Pixtral-12B-2409) and TheDrummer's UnslopNemo-12B-v3. The goal is to infuse as much Drummer goodness without breaking vision input. And it looks like it's worked!
## Usage
Needs more testing to identify best sampling params, but so far just using ~0.7 temp + 0.03 min p has been rock solid.
Use the included chat template (Mistral). No chatml support yet.
## Credits
- Mistral for [mistralai/Pixtral-12B-2409](https://huggingface.co/mistralai/Pixtral-12B-2409)
- Unsloth for [unsloth/Pixtral-12B-2409](https://huggingface.co/unsloth/Pixtral-12B-2409) transformers conversion
- TheDrummer for [TheDrummer/UnslopNemo-12B-v3](https://huggingface.co/TheDrummer/UnslopNemo-12B-v3)
## Available Sizes
| Repo | Bits | Head Bits | Size |
| ----------- | ------ | ------ | ------ |
| [nintwentydo/Razorback-12B-v0.2-exl2-4.0bpw](https://huggingface.co/nintwentydo/Razorback-12B-v0.2-exl2-4.0bpw) | 4.0 | 6.0 | 8.19 GB |
| [nintwentydo/Razorback-12B-v0.2-exl2-5.0bpw](https://huggingface.co/nintwentydo/Razorback-12B-v0.2-exl2-5.0bpw) | 5.0 | 6.0 | 9.54 GB |
| [nintwentydo/Razorback-12B-v0.2-exl2-6.0bpw](https://huggingface.co/nintwentydo/Razorback-12B-v0.2-exl2-6.0bpw) | 6.0 | 8.0 | 11.1 GB |
| [nintwentydo/Razorback-12B-v0.2-exl2-8.0bpw](https://huggingface.co/nintwentydo/Razorback-12B-v0.2-exl2-8.0bpw) | 8.0 | 8.0 | 13.7 GB |
|