I've been doing all my LoRA work on AMD hardware with Linux; I'm looking forward to your notes! I sometimes still do it on CPU because it's easy to renice the task priority so the foreground tasks stay snappy.
The main challenge I have is keeping a solid ROCm bitsandbytes install when other packages want updates.
Thank you! If you liked this, and want 14B models with even smoother command of French, I'm a huge fan of https://huggingface.co/jpacifico/Chocolatine-2-14B-Instruct-v2.0.3!
Right-sizing language models is something I'm really here for. I find that a 1.5B parameter model fronting simple questions from a backing RAG source that a larger model gradually works on is more scalable. Classic information sources and stores can be QA'd, and they don't have such huge energy footprints.
AI will work out better if we give humans, classic code, SLMs, and frontier LLMs the roles they're right-sized for, and ensure data privacy and individual dignity at every stage of the contract.
Okay, this has become a major component of how I build model_stocks that keep IFEVAL high even while merging distantly related models, and this is the reason for some TIES merges to "qwenvergify" models you might have seen.
Here's the basic idea:
https://www.arcee.ai/blog/use-mergekit-to-extract-lora-adapters-from-any-fine-tuned-model
But not as many models are inter-compatible for LoRAs as you'd expect, because there are minor variations in size among some important finetunes. I get the train tracks to a standard width, as it were, and make them intercompatible with the "qwenvergify" TIES merges between two models, weight 1.0 for the model of interest and weight 0.0 for any Qwenvergence or Lamarck model for the tiny bit of infill. You now have all models intercompatible for what is akin to a super-high-precision DELLA merge of the most significant parts of the model, the most IFEVAL-preserving parts of the model. A rank 512 adapter extracts around 30% of the most defining aspects of the model, but captures around 90% of its distinct performance. A rank 128 adapter captures around 8% of the model, but about 70% of its distinct performance.
I arrived at this while thinking about the implication of @rombodawg 's "Continuous Fine Tuning" strategy, and reading I-forget-which-arxiv-paper and I really need to find that again. It's like the opposite side of the coin from how rombodawg uses it. I use it at the beginning to get a large model_stock started. He uses it to extract most of your merge at the end and apply it to a target model to avoid catastrophic forgetting.
There. Now you know the methodology behind my merge YAML that produced https://huggingface.co/sometimesanotion/Qwenvergence-14B-v13-Prose-DS - or, the model that calls itself "Qwenconceited-14B-v13-DeepSuffering". ๐
Adapters from a strong IFEVAL+BBH model applied to the majority of the models in the model_stock merge, in a mixture of rank sizes between 32 and 128, get them on the same page for core operation. Applying a Virtuoso or Chocolatine-based LoRA to just any model out there could cause instability, but the model_stock smooths many varying levels of adapter merges out.
That's enough for you to digest for now, and @rombodawg might be interested to know he inspired such a different strategy from anything he's shared.
You can reach me on Discord, my username is as you'd expect.
Once I show you how Qwentinuum broke the barrier and finally got stabilized, and made Vimarckoso v3, you'll see why I'm being a little careful. It takes multiple steps to reliably tame weighty breadcrumbs merges, and I'm using Makefiles to make sure nothing gets skipped. That's not so easily posted to a modelcard! If people misuse parts of my recipe, especially with more CoT models out there, we'll get spammed with a lot of unstable models.
But the rewards of getting it right!
@Inschrift-Spruch-Raum
, I have a treat for you. I'm glad you liked v12 of Qwenvergence Prose - but I've struck gold. 13 just might be your lucky number!
https://huggingface.co/sometimesanotion/Qwenvergence-14B-v13-Prose-DS
I've really been pondering that, and it's almost certainly because of the blend of R1 and Krystalan/DRT-o1-14B. We have two different CoT lineages feeding into one model - wonderful, until it's not! DRT is a bit hard to give up. I think this is where we finally have done all we can do with merging, however fancy, and get down to fine-tuning, because if DRT and DS's influences sync up, it'll be magic.
I've spilled some of the beans in little separate doses, because I've hoped to prompt people to fill in the blanks with unique ideas rather than inspire a lot of copypasta. There's a lot of stuff that is just unique to my own workflow, but there's also some reaaaaally long and detailed YAML.
I do feel that what happens between the model_stock + LoRAs and the SLERP+TIES has been loosely described. It really is just a bit of general info about which layers influence what metric, like multiple gradients overlaid. I tend to keep densities under 40 or even 30, because if there's a strong core model, each extra model needs to leave headroom for the others.
Hit me up, though, I'm particularly grateful for your contribution!
While Arcee beats Lamarck 0.7 and
tempesthenno-ppo-ckpt40 for IFEVAL, BBH, and MATH, you score 23.55% higher on GPQA, 1.96% higher on MUSR, and 2.49% higher on MUSR than Virtuoso Small v2.
Plus, I'm thinking you fine-tune for use cases Arcee and I don't.
@Inschrift-Spruch-Raum is right. My estimates based off of comparator are only partially correct. Are percentile calculations on the leaderboard changing? Regardless, this graph shows why nobody needs to give up on their models, especially when each one's making a specialized contribution. Diversity is a benefit to us all.
I really like how this class of models proves that MUSR isn't just what you get when you throw IFEVAL into a blender. ๐
Wow! I have to check what I saw in comparator and based estimates off of. There's no doubt that Virtuoso Small v2 is a great model, and I'm already working on a Qwenvergence based on it. It's as awesome at IFEVAL and BBH as I'd thought.
Qwenvergence is the model_stock that makes the bases to blend in varying proportions across Lamarck's layers. Yet, it's not mere raw material. I'm getting really outstanding results out of Qwenvergence-14B-v12-Prose-DS's successor which includes Virtuoso Small v2. It's playing very nicely with the other components!
My high-benchmarking merges have included Virtuoso v1 at nearly every stage, and I am now creating a new generation switching in V2 where apt.
Feedback from finetuners suggests my minimal compute and Arcee's MergeKit have given them a shortcut to great results. Smart merging really is energy efficient. Thank you for helping us push the limits!
