bruhzair/Mutant-Writer-t5-123b

Mutant-t5

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the NuSLERP merge method.

Models Merged

The following models were included in the merge:

• /workspace/cache/models--gghfez--Writer-Large-2411-v2.1/snapshots/cb12d12db43e04ae42ac7a3c3ac987c867fd4757
• /workspace/cache/models--anthracite-org--magnum-v4-123b/snapshots/68fdd395bf5282429aa11d3b2737add1944243b3

Configuration

The following YAML configuration was used to produce this model:

dtype: bfloat16
merge_method: nuslerp
slices:
- sources:
  - layer_range: [0, 88]
    model: /workspace/cache/models--anthracite-org--magnum-v4-123b/snapshots/68fdd395bf5282429aa11d3b2737add1944243b3
    parameters:
      weight: [0.3, 0.4, 0.5, 0.6, 0.5, 0.4, 0.3]
  - layer_range: [0, 88]
    model: /workspace/cache/models--gghfez--Writer-Large-2411-v2.1/snapshots/cb12d12db43e04ae42ac7a3c3ac987c867fd4757
    parameters:
      weight: [0.7, 0.6, 0.5, 0.4, 0.5, 0.6, 0.7]