This release? No Deepseek R1
This release came out prior to the Deepseek R1 distillation, and Lamarck 0.7 included a small amount of R1 which boosted its MATH and MUSR. For reasons of possible lingering political bias, as well some instances of belabored reasoning that I believe come from combinations of different architectures and CoTs, I am now working on a Lamarck v0.8 and Lamarck v0.8 DS: two merges, one with and one without DeepSeek. I will keep the merge recipes as similar as possible otherwise.
Merging cross-architecture distills can have its surprises! Arcee AI's Medius and its distill from Llama is in Lamarck's ancestry, if you go back far enough. This means we have Qwen, Llama, and Deepseek with international contributions. I will try to give finetuners a little more choice of starting points.