---
license: cc-by-nc-4.0
---
**Calibration dataset**: VMware Open Instruct, 4096
<p align="center"><font size="7"> <b>Dumb assistant, 🔥 author.</b></font></p>
<p align="center"><font size="5"> <b>4-bit Examples with Alpaca</b></font></p>
<p align="center"><img src="https://i.postimg.cc/VNXpwDCY/Screenshot-2023-12-06-212944.png"/>
<p align="center"><img src="https://i.ibb.co/TPDsD2z/Screenshot-2023-12-06-211216.png"/>
<p align="center"><font size="5"><b><a href="https://i.ibb.co/4PhnXjK/Screenshot-2023-12-06-205831.png">!!NSFW!! - 🔥Erotica Writing Example🔥 - !!NSFW!!</font></a></b></p>

Thanks to [Charles Goddard](https://huggingface.co/chargoddard) for the recipe.

The idea here is to "move" Iambe from being based on vanilla L2 to being based on sequelbox/DynamicFactor instead.

Because task_arithmetic uses the raw deltas, this should be similar to if the SFT had been done over DynamicFactor.

## Recipe

merge_method: task_arithmetic

base_model: athirdpath/BigLlama-20b-v1.1      # Base model you want to "move out" from

models:

  - model: athirdpath/Iambe-20b-DARE-v2       # SFTd model you want to transfer

  - model: athirdpath/DoubleFactor-20b      # Base model you want to "move in" to

parameters:

  - weight: 1.0

dtype: bfloat16