File size: 1,213 Bytes
845e50f a322625 1746de6 665bba0 1746de6 476b93b e432f9c 476b93b e432f9c 476b93b e432f9c 476b93b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
---
license: cc-by-nc-4.0
---
<p align="center"><font size="7"> <b>Dumb assistant, 🔥 author.</b></font></p>
<p align="center"><font size="5"> <b>4-bit Examples with Alpaca</b></font></p>
<p align="center"><img src="https://i.postimg.cc/VNXpwDCY/Screenshot-2023-12-06-212944.png"/>
<p align="center"><img src="https://i.ibb.co/TPDsD2z/Screenshot-2023-12-06-211216.png"/>
<p align="center"><font size="5"><b><a href="https://i.ibb.co/4PhnXjK/Screenshot-2023-12-06-205831.png">!!NSFW!! - 🔥Erotica Writing Example🔥 - !!NSFW!!</font></a></b></p>
Thanks to [Charles Goddard](https://huggingface.co/chargoddard) for the recipe.
The idea here is to "move" Iambe from being based on vanilla L2 to being based on sequelbox/DynamicFactor instead.
Because task_arithmetic uses the raw deltas, this should be similar to if the SFT had been done over DynamicFactor.
## Recipe
merge_method: task_arithmetic
base_model: athirdpath/BigLlama-20b-v1.1 # Base model you want to "move out" from
models:
- model: athirdpath/Iambe-20b-DARE-v2 # SFTd model you want to transfer
- model: athirdpath/DoubleFactor-20b # Base model you want to "move in" to
parameters:
- weight: 1.0
dtype: bfloat16 |