nisten commited on
Commit
e885683
1 Parent(s): 7116803

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -3
README.md CHANGED
@@ -11,6 +11,7 @@ base_model: [deepseek-ai/DeepSeek-V2-Chat-0628]
11
  >🐧 On Linux `sudo apt install -y aria2`
12
  >
13
  >🍎 On Mac `brew install aria2`
 
14
  >These links will download 9x faster, feel free to paste them all in or one at a time
15
 
16
  ```verilog
@@ -175,15 +176,30 @@ aria2c -x 8 -o deepseek-0628-q4_0_8_8_faster_dumber-00003-of-00004.gguf \
175
 
176
  aria2c -x 8 -o deepseek-0628-q4_0_8_8_faster_dumber-00004-of-00004.gguf \
177
  https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek-0628-q4_0_8_8_faster_dumber-00004-of-00004.gguf
 
 
 
 
 
 
 
 
 
178
 
179
- # IQ1M version (78.7GB total)
 
180
  aria2c -x 8 -o deepseek_0628_cpu-iq1m-00001-of-00002.gguf \
181
  https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek_0628_cpu-iq1m-00001-of-00002.gguf
182
 
183
  aria2c -x 8 -o deepseek_0628_cpu-iq1m-00002-of-00002.gguf \
184
  https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek_0628_cpu-iq1m-00002-of-00002.gguf
185
-
186
- # IQ1_S version (62.7GB total)
 
 
 
 
 
187
  aria2c -x 8 -o deepseek_0628_cpu_iq1_s-00001-of-00002.gguf \
188
  https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek_0628_cpu_iq1_s-00001-of-00002.gguf
189
 
 
11
  >🐧 On Linux `sudo apt install -y aria2`
12
  >
13
  >🍎 On Mac `brew install aria2`
14
+ >
15
  >These links will download 9x faster, feel free to paste them all in or one at a time
16
 
17
  ```verilog
 
176
 
177
  aria2c -x 8 -o deepseek-0628-q4_0_8_8_faster_dumber-00004-of-00004.gguf \
178
  https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek-0628-q4_0_8_8_faster_dumber-00004-of-00004.gguf
179
+ ```
180
+
181
+ The following 1 bit mixed quant versions are strangely good:
182
+
183
+ <figure>
184
+ <img src="https://cdn-uploads.huggingface.co/production/uploads/6379683a81c1783a4a2ddba8/Qxx4p2l0prHiScCdL68XK.png" alt="deepseek_0628_cpu-iq1m example response">
185
+ <figcaption><strong>deepseek_0628_cpu-iq1m (73.27 GB):</strong> example response is strangely good</figcaption>
186
+ </figure>
187
+
188
 
189
+ ```bash
190
+ # IQ1M version (73.27 GB)
191
  aria2c -x 8 -o deepseek_0628_cpu-iq1m-00001-of-00002.gguf \
192
  https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek_0628_cpu-iq1m-00001-of-00002.gguf
193
 
194
  aria2c -x 8 -o deepseek_0628_cpu-iq1m-00002-of-00002.gguf \
195
  https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek_0628_cpu-iq1m-00002-of-00002.gguf
196
+ ```
197
+ <figure>
198
+ <img src="https://cdn-uploads.huggingface.co/production/uploads/6379683a81c1783a4a2ddba8/N0lQ5HAJbNbXIG1MbtB4x.png" alt="deepseek_0628_cpu-iq1s example response">
199
+ <figcaption><strong>deepseek_0628_cpu-iq1s (58.42 GB):</strong> Even the IQ1_S version (52.7GB total)</figcaption>
200
+ </figure>
201
+ ```bash
202
+ # IQ1_S version (58.42 GB)
203
  aria2c -x 8 -o deepseek_0628_cpu_iq1_s-00001-of-00002.gguf \
204
  https://huggingface.co/nisten/deepseek-0628-gguf/resolve/main/deepseek_0628_cpu_iq1_s-00001-of-00002.gguf
205