pszemraj

/

Llama-3-6.3b-v0.1

like 6

Thanks! I'd say it's pretty performant, but IMO it's best to think of this as a tradeoff between speed and performance. You get some memory savings from the reduced number of params, but the main benefit is faster training and inference from removing 6 (~20%) of the layers.

Note that this is the base model-not the instruct-so it needs some fine-tuning before practical use.