Update README.md
Browse files
README.md
CHANGED
@@ -11,6 +11,7 @@ base_model: [deepseek-ai/DeepSeek-V2-Chat-0628]
|
|
11 |
>🐧 On Linux `sudo apt install -y aria2`
|
12 |
>
|
13 |
>🍎 On Mac `brew install aria2`
|
|
|
14 |
>These links will download 9x faster, feel free to paste them all in or one at a time
|
15 |
|
16 |
```verilog
|
@@ -175,15 +176,30 @@ aria2c -x 8 -o deepseek-0628-q4_0_8_8_faster_dumber-00003-of-00004.gguf \
|
|
175 |
|
176 |
aria2c -x 8 -o deepseek-0628-q4_0_8_8_faster_dumber-00004-of-00004.gguf \
|
177 |
https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek-0628-q4_0_8_8_faster_dumber-00004-of-00004.gguf
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
178 |
|
179 |
-
|
|
|
180 |
aria2c -x 8 -o deepseek_0628_cpu-iq1m-00001-of-00002.gguf \
|
181 |
https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek_0628_cpu-iq1m-00001-of-00002.gguf
|
182 |
|
183 |
aria2c -x 8 -o deepseek_0628_cpu-iq1m-00002-of-00002.gguf \
|
184 |
https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek_0628_cpu-iq1m-00002-of-00002.gguf
|
185 |
-
|
186 |
-
|
|
|
|
|
|
|
|
|
|
|
187 |
aria2c -x 8 -o deepseek_0628_cpu_iq1_s-00001-of-00002.gguf \
|
188 |
https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek_0628_cpu_iq1_s-00001-of-00002.gguf
|
189 |
|
|
|
11 |
>🐧 On Linux `sudo apt install -y aria2`
|
12 |
>
|
13 |
>🍎 On Mac `brew install aria2`
|
14 |
+
>
|
15 |
>These links will download 9x faster, feel free to paste them all in or one at a time
|
16 |
|
17 |
```verilog
|
|
|
176 |
|
177 |
aria2c -x 8 -o deepseek-0628-q4_0_8_8_faster_dumber-00004-of-00004.gguf \
|
178 |
https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek-0628-q4_0_8_8_faster_dumber-00004-of-00004.gguf
|
179 |
+
```
|
180 |
+
|
181 |
+
The following 1 bit mixed quant versions are strangely good:
|
182 |
+
|
183 |
+
<figure>
|
184 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6379683a81c1783a4a2ddba8/Qxx4p2l0prHiScCdL68XK.png" alt="deepseek_0628_cpu-iq1m example response">
|
185 |
+
<figcaption><strong>deepseek_0628_cpu-iq1m (73.27 GB):</strong> example response is strangely good</figcaption>
|
186 |
+
</figure>
|
187 |
+
|
188 |
|
189 |
+
```bash
|
190 |
+
# IQ1M version (73.27 GB)
|
191 |
aria2c -x 8 -o deepseek_0628_cpu-iq1m-00001-of-00002.gguf \
|
192 |
https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek_0628_cpu-iq1m-00001-of-00002.gguf
|
193 |
|
194 |
aria2c -x 8 -o deepseek_0628_cpu-iq1m-00002-of-00002.gguf \
|
195 |
https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek_0628_cpu-iq1m-00002-of-00002.gguf
|
196 |
+
```
|
197 |
+
<figure>
|
198 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6379683a81c1783a4a2ddba8/N0lQ5HAJbNbXIG1MbtB4x.png" alt="deepseek_0628_cpu-iq1s example response">
|
199 |
+
<figcaption><strong>deepseek_0628_cpu-iq1s (58.42 GB):</strong> Even the IQ1_S version (52.7GB total)</figcaption>
|
200 |
+
</figure>
|
201 |
+
```bash
|
202 |
+
# IQ1_S version (58.42 GB)
|
203 |
aria2c -x 8 -o deepseek_0628_cpu_iq1_s-00001-of-00002.gguf \
|
204 |
https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek_0628_cpu_iq1_s-00001-of-00002.gguf
|
205 |
|