Hugging Face's logo

XelotX
/
Midnight-Miqu-70B-v1.5-XelotX-Quants

Transformers
GGUF
English
mergekit
Merge
Inference Endpoints
conversational
Model card Files Community
XelotX mradermacher commited on
Commit
3a26a9b
·
verified ·
0 Parent(s):

Duplicate from mradermacher/Midnight-Miqu-70B-v1.5-GGUF

Browse files

Co-authored-by: Michael Radermacher <[email protected]>

Files changed (18) hide show
  1. .gitattributes +51 -0
  2. Midnight-Miqu-70B-v1.5.IQ3_M.gguf +3 -0
  3. Midnight-Miqu-70B-v1.5.IQ3_S.gguf +3 -0
  4. Midnight-Miqu-70B-v1.5.IQ3_XS.gguf +3 -0
  5. Midnight-Miqu-70B-v1.5.IQ4_XS.gguf +3 -0
  6. Midnight-Miqu-70B-v1.5.Q2_K.gguf +3 -0
  7. Midnight-Miqu-70B-v1.5.Q3_K_L.gguf +3 -0
  8. Midnight-Miqu-70B-v1.5.Q3_K_M.gguf +3 -0
  9. Midnight-Miqu-70B-v1.5.Q3_K_S.gguf +3 -0
  10. Midnight-Miqu-70B-v1.5.Q4_K_M.gguf +3 -0
  11. Midnight-Miqu-70B-v1.5.Q4_K_S.gguf +3 -0
  12. Midnight-Miqu-70B-v1.5.Q5_K_M.gguf +3 -0
  13. Midnight-Miqu-70B-v1.5.Q5_K_S.gguf +3 -0
  14. Midnight-Miqu-70B-v1.5.Q6_K.gguf.part1of2 +3 -0
  15. Midnight-Miqu-70B-v1.5.Q6_K.gguf.part2of2 +3 -0
  16. Midnight-Miqu-70B-v1.5.Q8_0.gguf.part1of2 +3 -0
  17. Midnight-Miqu-70B-v1.5.Q8_0.gguf.part2of2 +3 -0
  18. README.md +64 -0
.gitattributes ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Midnight-Miqu-70B-v1.5.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Midnight-Miqu-70B-v1.5.Q8_0.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
38
+ Midnight-Miqu-70B-v1.5.Q8_0.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
39
+ Midnight-Miqu-70B-v1.5.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Midnight-Miqu-70B-v1.5.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Midnight-Miqu-70B-v1.5.Q6_K.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
42
+ Midnight-Miqu-70B-v1.5.Q6_K.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
43
+ Midnight-Miqu-70B-v1.5.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Midnight-Miqu-70B-v1.5.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Midnight-Miqu-70B-v1.5.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Midnight-Miqu-70B-v1.5.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Midnight-Miqu-70B-v1.5.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Midnight-Miqu-70B-v1.5.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Midnight-Miqu-70B-v1.5.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Midnight-Miqu-70B-v1.5.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Midnight-Miqu-70B-v1.5.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Midnight-Miqu-70B-v1.5.IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1913b389a78344251e559ff4c820c85b52ead68d6e01bc4f64523034eafcd517
3
+ size 31253526080
Midnight-Miqu-70B-v1.5.IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17201b8a08e3b5d7a8b150230231c7836e198af341cb23d41b60aee35010d47b
3
+ size 30228543040
Midnight-Miqu-70B-v1.5.IQ3_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f225ece1fd2e53cc29072f1ead4b67456e69cc4e29ab273e6ec80ba9794590a
3
+ size 28451206720
Midnight-Miqu-70B-v1.5.IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:71d03e690d444933f76a43bba7c22c294a7ba7b4af09fdad526981d7a1de14aa
3
+ size 37506070080
Midnight-Miqu-70B-v1.5.Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3026b638243457a7610f4d1ef69d9e6bbb8e6ae26a499aa40fd1213d74ee42a6
3
+ size 25771685440
Midnight-Miqu-70B-v1.5.Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1bdf23d9c1695c85a3238f3eddce6b8512121e95ee3e3993df6e81545386b489
3
+ size 36457084480
Midnight-Miqu-70B-v1.5.Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c5080c429a53b06f1c7f9efbe3f4399992de76831dc39d18e4a1c53006b139e
3
+ size 33583986240
Midnight-Miqu-70B-v1.5.Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c841cd9f1ec0c74c9b829eeac7534087119735fbb0172674c4c52186383dcfeb
3
+ size 30228543040
Midnight-Miqu-70B-v1.5.Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0ab2054889a92bcfed73d9dbf58750564bdcb398bf610616f0f703791b4cb188
3
+ size 41732159040
Midnight-Miqu-70B-v1.5.Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f881892e5ce8d4e6ae7e953062ed6a1f1fa37908374c720ce370c014d4a024e0
3
+ size 39558985280
Midnight-Miqu-70B-v1.5.Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c654f178ecaf565a3f40df07c1363daed7e59cdffa5ffdb7cd5318cb587cba4b
3
+ size 49063016000
Midnight-Miqu-70B-v1.5.Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aeb126e4a90a76abb335467afa80220c9d7376b13cfb1f84042fc7da30d6a938
3
+ size 47770646080
Midnight-Miqu-70B-v1.5.Q6_K.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:027058b43f7247ad1b1f8568a1770b208638db9ab58213f1ef795b7d642c6907
3
+ size 28991029248
Midnight-Miqu-70B-v1.5.Q6_K.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f56f99e0ba4efaf0e70755006682d887c3b8e4c95e688d4be8ce7192eecf5898
3
+ size 27905586752
Midnight-Miqu-70B-v1.5.Q8_0.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17f1580deb625159f35d64900b3df9445dff9a08353b92a7fb50d060356d139e
3
+ size 37580963840
Midnight-Miqu-70B-v1.5.Q8_0.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d229abf5629ad05ee1a8ac6443e7a3ee2e716393a27516747f9c20333845931
3
+ size 35957126720
README.md ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: sophosympatheia/Midnight-Miqu-70B-v1.5
3
+ language:
4
+ - en
5
+ library_name: transformers
6
+ license: other
7
+ quantized_by: mradermacher
8
+ tags:
9
+ - mergekit
10
+ - merge
11
+ ---
12
+ ## About
13
+
14
+ static quants of https://huggingface.co/sophosympatheia/Midnight-Miqu-70B-v1.5
15
+
16
+ <!-- provided-files -->
17
+ weighted/imatrix quants are available at https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF
18
+ ## Usage
19
+
20
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
21
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
22
+ more details, including on how to concatenate multi-part files.
23
+
24
+ ## Provided Quants
25
+
26
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
27
+
28
+ | Link | Type | Size/GB | Notes |
29
+ |:-----|:-----|--------:|:------|
30
+ | [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.Q2_K.gguf) | Q2_K | 25.9 | |
31
+ | [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.IQ3_XS.gguf) | IQ3_XS | 28.6 | |
32
+ | [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.IQ3_S.gguf) | IQ3_S | 30.3 | beats Q3_K* |
33
+ | [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.Q3_K_S.gguf) | Q3_K_S | 30.3 | |
34
+ | [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.IQ3_M.gguf) | IQ3_M | 31.4 | |
35
+ | [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.Q3_K_M.gguf) | Q3_K_M | 33.7 | lower quality |
36
+ | [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.Q3_K_L.gguf) | Q3_K_L | 36.6 | |
37
+ | [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.IQ4_XS.gguf) | IQ4_XS | 37.6 | |
38
+ | [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.Q4_K_S.gguf) | Q4_K_S | 39.7 | fast, recommended |
39
+ | [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.Q4_K_M.gguf) | Q4_K_M | 41.8 | fast, recommended |
40
+ | [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.Q5_K_S.gguf) | Q5_K_S | 47.9 | |
41
+ | [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.Q5_K_M.gguf) | Q5_K_M | 49.2 | |
42
+ | [PART 1](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.Q6_K.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.Q6_K.gguf.part2of2) | Q6_K | 57.0 | very good quality |
43
+ | [PART 1](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.Q8_0.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.Q8_0.gguf.part2of2) | Q8_0 | 73.6 | fast, best quality |
44
+
45
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
46
+ types (lower is better):
47
+
48
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
49
+
50
+ And here are Artefact2's thoughts on the matter:
51
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
52
+
53
+ ## FAQ / Model Request
54
+
55
+ See https://huggingface.co/mradermacher/model_requests for some answers to
56
+ questions you might have and/or if you want some other model quantized.
57
+
58
+ ## Thanks
59
+
60
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
61
+ me use its servers and providing upgrades to my workstation to enable
62
+ this work in my free time.
63
+
64
+ <!-- end -->