Update README.md

README.md

@@ -22,6 +22,7 @@ This is NOT your usual frankenmerge created using [mergekit](https://github.com/
 
 This model was merged using the passthrough merge method, but employing a variation of the Block Expansion method described in the paper [LLaMA Pro: Progressive LLaMA with Block Expansion](https://arxiv.org/abs/2401.02415).
 The authors of the paper added new layers interleaved in between the original layers of the model, setting the parameters of the o_proj and down_proj layers to zero. This effectively adds layers that will just output their input allowing the model to remain functional even without further training. These new layers can than be target during training or fine-tuning without risking catastrophic forgetting, if you follow the author's training method to freeze the original layers and only train the new layers.
+
 I used the same method but added the new layers to the end of the model. My rationale is that the level of abstraction increases with each layer of the model. So, while new layers spread along the original layers will help the model to learn new tasks, adding layers to the end of the model and then re-training/fine-tuning the model on tasks it already performs well could improve the models understanding of those task and perform than better.
 These model has not yet received additional training, so it should perform close to the original model.