anakin87 commited on
Commit
fbbe7d6
โ€ข
1 Parent(s): 84fd9ee

training notebook

Browse files
Files changed (2) hide show
  1. README.md +7 -3
  2. notebooks/training.ipynb +0 -0
README.md CHANGED
@@ -23,7 +23,11 @@ Fine-tuned version of [Microsoft/Phi-3.5-mini-instruct](https://huggingface.co/m
23
  - Small yet powerful model with 3.82 billion parameters
24
  - Supports 128k context length
25
 
26
- [๐Ÿ’ฌ๐Ÿ‡ฎ๐Ÿ‡น Chat with the model on Hugging Face Spaces](https://huggingface.co/spaces/anakin87/Phi-3.5-mini-ITA)
 
 
 
 
27
 
28
  ## ๐Ÿ† Evaluation
29
 
@@ -112,6 +116,6 @@ It underwent 2 epochs of instruction fine-tuning on the [FineTome-100k](https://
112
  I adopted a relatively new technique for parameter-efficient learning: [Spectrum](https://arxiv.org/abs/2406.06623).
113
  The idea is to train only the layers of the model with high Signal-to-Noise Ratio (SNR) and โ„๏ธ freeze the rest.
114
 
115
- Training required about 14 hours on a single A40 GPU.
116
 
117
- I may release a guide/tutorial soon. Stay tuned! ๐Ÿ“ป
 
23
  - Small yet powerful model with 3.82 billion parameters
24
  - Supports 128k context length
25
 
26
+ - [๐Ÿ’ฌ๐Ÿ‡ฎ๐Ÿ‡น Chat with the model on Hugging Face Spaces](https://huggingface.co/spaces/anakin87/Phi-3.5-mini-ITA)
27
+ - [GGUF quants](https://huggingface.co/QuantFactory/Phi-3.5-mini-ITA-GGUF)
28
+
29
+ ๐Ÿ‹๏ธโ€โ™‚๏ธ **Do you want to understand how the model was trained?**
30
+ Check out the [๐Ÿ“– full walkthrough article](https://huggingface.co/blog/anakin87/spectrum) and the accompanying [๐Ÿ’ป notebook](./notebooks/training.ipynb)
31
 
32
  ## ๐Ÿ† Evaluation
33
 
 
116
  I adopted a relatively new technique for parameter-efficient learning: [Spectrum](https://arxiv.org/abs/2406.06623).
117
  The idea is to train only the layers of the model with high Signal-to-Noise Ratio (SNR) and โ„๏ธ freeze the rest.
118
 
119
+ Training required about 14 hours on a single A6000 GPU.
120
 
121
+ **For complete training details**, check out the [๐Ÿ“– full walkthrough article](https://huggingface.co/blog/anakin87/spectrum) and the accompanying [๐Ÿ’ป notebook](./notebooks/training.ipynb).
notebooks/training.ipynb ADDED
The diff for this file is too large to render. See raw diff