--- base_model: - unsloth/Llama-3.2-3B - FuseAI/FuseChat-Llama-3.2-3B-Instruct library_name: transformers tags: - mergekit - merge --- # merge This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## Merge Details ### Merge Method This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [unsloth/Llama-3.2-3B](https://huggingface.co/unsloth/Llama-3.2-3B) as a base. ### Models Merged The following models were included in the merge: * [FuseAI/FuseChat-Llama-3.2-3B-Instruct](https://huggingface.co/FuseAI/FuseChat-Llama-3.2-3B-Instruct) ### Configuration The following YAML configuration was used to produce this model: ```yaml models: - model: unsloth/Llama-3.2-3B parameters: weight: 0.5 density: 0.7 - model: FuseAI/FuseChat-Llama-3.2-3B-Instruct parameters: weight: 0.5 density: 0.6 merge_method: ties base_model: unsloth/Llama-3.2-3B parameters: normalize: true int8_mask: true dtype: bfloat16 tokenizer_source: unsloth/Llama-3.2-3B ```