---
base_model:
- sometimesanotion/Lamarck-14B-v0.6
- sthenno-com/miscii-14b-1225
- sometimesanotion/Qwentinuum-14B-v013
- Krystalan/DRT-o1-14B
- sometimesanotion/Qwenvergence-14B-v3-Prose
- arcee-ai/Virtuoso-Small
- huihui-ai/Qwen2.5-14B-Instruct-abliterated-v2
library_name: transformers
tags:
- mergekit
- merge
license: apache-2.0
language:
- en
metrics:
- accuracy
pipeline_tag: text-generation
---
This isn't meant for end users.  This merge is meant to expand the range of weights and scores possible to a standardized base, and may be showing signs of overfitting.  I'm leaving it public because it's made new record scores for MUSR and MATH for 14B Qwen2.5 models, and that's worth studying.

This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using sometimesanotion/Lamarck-14B-v0.7-Base-001 as a base.

### Models Merged

The following models were included in the merge:
* [sthenno-com/miscii-14b-1225](https://huggingface.co/sthenno-com/miscii-14b-1225)
* [sometimesanotion/Qwentinuum-14B-v013](https://huggingface.co/sometimesanotion/Qwentinuum-14B-v013)
* [Krystalan/DRT-o1-14B](https://huggingface.co/Krystalan/DRT-o1-14B)
* [sometimesanotion/Qwenvergence-14B-v3-Prose](https://huggingface.co/sometimesanotion/Qwenvergence-14B-v3-Prose)
* [arcee-ai/Virtuoso-Small](https://huggingface.co/arcee-ai/Virtuoso-Small)
* [sometimesanotion/Lamarck-14B-v0.6](https://huggingface.co/sometimesanotion/Lamarck-14B-v0.6)
* [sometimesanotion/Abliterate-Qwenvergence](https://huggingface.co/sometimesanotion/Abliterate-Qwenvergence)

### Configuration

The following YAML configuration was used to produce this model (**note**:  -qv suffixes indicate LoRA application from [Abliterate-Qwenvergence](https://huggingface.co/sometimesanotion/Abliterate-Qwenvergence) which is almost identical to [huihui-ai/Qwen2.5-14B-Instruct-abliterated-v2](https://huggingface.co/huihui-ai/Qwen2.5-14B-Instruct-abliterated-v2))

```yaml
name:                Qwenvergence-14B-v9
merge_method:        model_stock
base_model:          sometimesanotion/Lamarck-14B-v0.7-Base-001
tokenizer_source:    sometimesanotion/Abliterate-Qwenvergence
dtype:               float32
out_dtype:           bfloat16
parameters:
  int8_mask:         true
  normalize:         true
  rescale:           false
models:
  - model:           sometimesanotion/Lamarck-14B-v0.6
  - model:           sometimesanotion/Qwenvergence-14B-v3-Prose-qv256
  - model:           Krystalan/DRT-o1-14B-qv128
  - model:           arcee-ai/Virtuoso-Small-qv64
  - model:           sometimesanotion/Lamarck-14B-v0.6
  - model:           sometimesanotion/Qwentinuum-14B-v013-qv512
  - model:           sthenno-com/miscii-14b-1225-qv64
  - model:           sometimesanotion/Qwenvergence-14B-v3-Prose-qv256
  - model:           sometimesanotion/Lamarck-14B-v0.6

```