My Thalassic series includes my favorite Llama 3 models with Deepseek R1, using various methods I made 4 merges. Thalassic Delta, which I made with an SCE merge, was the best of the set. I actually got some advice from Steelskull about how I had my top k parameter way too high. So I decided to lower it for this merge. Now because I feel Delta was still a decent model with the top k on 1, I halved the top k parameter to 0.50 which is basically the max recommended setting.
merge
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the SCE merge method using SicariusSicariiStuff/Negative_LLAMA_70B as a base.
Models Merged
The following models were included in the merge:
- deepseek-ai/DeepSeek-R1-Distill-Llama-70B
- Sao10K/L3.1-70B-Hanami-x1
- TheDrummer/Anubis-70B-v1
- EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1
- Sao10K/70B-L3.3-Cirrus-x1
Configuration
The following YAML configuration was used to produce this model:
models:
# Pivot model
- model: deepseek-ai/DeepSeek-R1-Distill-Llama-70B
# Target models
- model: Sao10K/70B-L3.3-Cirrus-x1
- model: Sao10K/L3.1-70B-Hanami-x1
- model: TheDrummer/Anubis-70B-v1
- model: EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1
merge_method: sce
base_model: SicariusSicariiStuff/Negative_LLAMA_70B
parameters:
select_topk: 0.5
dtype: bfloat16
- Downloads last month
- 1