training notebook
Browse files- README.md +7 -3
- notebooks/training.ipynb +0 -0
README.md
CHANGED
@@ -23,7 +23,11 @@ Fine-tuned version of [Microsoft/Phi-3.5-mini-instruct](https://huggingface.co/m
|
|
23 |
- Small yet powerful model with 3.82 billion parameters
|
24 |
- Supports 128k context length
|
25 |
|
26 |
-
[๐ฌ๐ฎ๐น Chat with the model on Hugging Face Spaces](https://huggingface.co/spaces/anakin87/Phi-3.5-mini-ITA)
|
|
|
|
|
|
|
|
|
27 |
|
28 |
## ๐ Evaluation
|
29 |
|
@@ -112,6 +116,6 @@ It underwent 2 epochs of instruction fine-tuning on the [FineTome-100k](https://
|
|
112 |
I adopted a relatively new technique for parameter-efficient learning: [Spectrum](https://arxiv.org/abs/2406.06623).
|
113 |
The idea is to train only the layers of the model with high Signal-to-Noise Ratio (SNR) and โ๏ธ freeze the rest.
|
114 |
|
115 |
-
Training required about 14 hours on a single
|
116 |
|
117 |
-
|
|
|
23 |
- Small yet powerful model with 3.82 billion parameters
|
24 |
- Supports 128k context length
|
25 |
|
26 |
+
- [๐ฌ๐ฎ๐น Chat with the model on Hugging Face Spaces](https://huggingface.co/spaces/anakin87/Phi-3.5-mini-ITA)
|
27 |
+
- [GGUF quants](https://huggingface.co/QuantFactory/Phi-3.5-mini-ITA-GGUF)
|
28 |
+
|
29 |
+
๐๏ธโโ๏ธ **Do you want to understand how the model was trained?**
|
30 |
+
Check out the [๐ full walkthrough article](https://huggingface.co/blog/anakin87/spectrum) and the accompanying [๐ป notebook](./notebooks/training.ipynb)
|
31 |
|
32 |
## ๐ Evaluation
|
33 |
|
|
|
116 |
I adopted a relatively new technique for parameter-efficient learning: [Spectrum](https://arxiv.org/abs/2406.06623).
|
117 |
The idea is to train only the layers of the model with high Signal-to-Noise Ratio (SNR) and โ๏ธ freeze the rest.
|
118 |
|
119 |
+
Training required about 14 hours on a single A6000 GPU.
|
120 |
|
121 |
+
**For complete training details**, check out the [๐ full walkthrough article](https://huggingface.co/blog/anakin87/spectrum) and the accompanying [๐ป notebook](./notebooks/training.ipynb).
|
notebooks/training.ipynb
ADDED
The diff for this file is too large to render.
See raw diff
|
|