Hugging Face's logo

bartowski
/
TableLLM-13b-GGUF

Text Generation
GGUF
English
Table
QA
Code
Inference Endpoints
Model card Files Community
bartowski commited on
Commit
4de7e86
•
1 Parent(s): 86dfb29

Llamacpp quants

Browse files
Files changed (18) hide show
  1. .gitattributes +16 -0
  2. README.md +42 -0
  3. TableLLM-13b-IQ3_M.gguf +3 -0
  4. TableLLM-13b-IQ3_S.gguf +3 -0
  5. TableLLM-13b-IQ4_NL.gguf +3 -0
  6. TableLLM-13b-IQ4_XS.gguf +3 -0
  7. TableLLM-13b-Q2_K.gguf +3 -0
  8. TableLLM-13b-Q3_K_L.gguf +3 -0
  9. TableLLM-13b-Q3_K_M.gguf +3 -0
  10. TableLLM-13b-Q3_K_S.gguf +3 -0
  11. TableLLM-13b-Q4_0.gguf +3 -0
  12. TableLLM-13b-Q4_K_M.gguf +3 -0
  13. TableLLM-13b-Q4_K_S.gguf +3 -0
  14. TableLLM-13b-Q5_0.gguf +3 -0
  15. TableLLM-13b-Q5_K_M.gguf +3 -0
  16. TableLLM-13b-Q5_K_S.gguf +3 -0
  17. TableLLM-13b-Q6_K.gguf +3 -0
  18. TableLLM-13b-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ TableLLM-13b-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ TableLLM-13b-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ TableLLM-13b-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
39
+ TableLLM-13b-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ TableLLM-13b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ TableLLM-13b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ TableLLM-13b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ TableLLM-13b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ TableLLM-13b-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
45
+ TableLLM-13b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ TableLLM-13b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ TableLLM-13b-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ TableLLM-13b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ TableLLM-13b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ TableLLM-13b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
51
+ TableLLM-13b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: llama2
3
+ datasets:
4
+ - RUCKBReasoning/TableLLM-SFT
5
+ language:
6
+ - en
7
+ tags:
8
+ - Table
9
+ - QA
10
+ - Code
11
+ quantized_by: bartowski
12
+ pipeline_tag: text-generation
13
+ ---
14
+
15
+ ## Llamacpp Quantizations of TableLLM-13b
16
+
17
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2589">b2589</a> for quantization.
18
+
19
+ Original model: https://huggingface.co/RUCKBReasoning/TableLLM-13b
20
+
21
+ Download a file (not the whole branch) from below:
22
+
23
+ | Filename | Quant type | File Size | Description |
24
+ | -------- | ---------- | --------- | ----------- |
25
+ | [TableLLM-13b-Q8_0.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-Q8_0.gguf) | Q8_0 | 13.83GB | Extremely high quality, generally unneeded but max available quant. |
26
+ | [TableLLM-13b-Q6_K.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-Q6_K.gguf) | Q6_K | 10.67GB | Very high quality, near perfect, *recommended*. |
27
+ | [TableLLM-13b-Q5_K_M.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-Q5_K_M.gguf) | Q5_K_M | 9.23GB | High quality, very usable. |
28
+ | [TableLLM-13b-Q5_K_S.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-Q5_K_S.gguf) | Q5_K_S | 8.97GB | High quality, very usable. |
29
+ | [TableLLM-13b-Q5_0.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-Q5_0.gguf) | Q5_0 | 8.97GB | High quality, older format, generally not recommended. |
30
+ | [TableLLM-13b-Q4_K_M.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-Q4_K_M.gguf) | Q4_K_M | 7.86GB | Good quality, uses about 4.83 bits per weight. |
31
+ | [TableLLM-13b-Q4_K_S.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-Q4_K_S.gguf) | Q4_K_S | 7.42GB | Slightly lower quality with small space savings. |
32
+ | [TableLLM-13b-IQ4_NL.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-IQ4_NL.gguf) | IQ4_NL | 7.41GB | Decent quality, similar to Q4_K_S, new method of quanting, |
33
+ | [TableLLM-13b-IQ4_XS.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-IQ4_XS.gguf) | IQ4_XS | 7.01GB | Decent quality, new method with similar performance to Q4. |
34
+ | [TableLLM-13b-Q4_0.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-Q4_0.gguf) | Q4_0 | 7.36GB | Decent quality, older format, generally not recommended. |
35
+ | [TableLLM-13b-Q3_K_L.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-Q3_K_L.gguf) | Q3_K_L | 6.92GB | Lower quality but usable, good for low RAM availability. |
36
+ | [TableLLM-13b-Q3_K_M.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-Q3_K_M.gguf) | Q3_K_M | 6.33GB | Even lower quality. |
37
+ | [TableLLM-13b-IQ3_M.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-IQ3_M.gguf) | IQ3_M | 5.98GB | Medium-low quality, new method with decent performance. |
38
+ | [TableLLM-13b-IQ3_S.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-IQ3_S.gguf) | IQ3_S | 5.65GB | Lower quality, new method with decent performance, recommended over Q3 quants. |
39
+ | [TableLLM-13b-Q3_K_S.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-Q3_K_S.gguf) | Q3_K_S | 5.65GB | Low quality, not recommended. |
40
+ | [TableLLM-13b-Q2_K.gguf](https://huggingface.co/bartowski/TableLLM-13b-GGUF/blob/main/TableLLM-13b-Q2_K.gguf) | Q2_K | 4.85GB | Extremely low quality, *not* recommended.
41
+
42
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
TableLLM-13b-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:053cf7e5ac21083680760d4381baec8a6e507261c16a60f1329e7129c34db4d6
3
+ size 5984612896
TableLLM-13b-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:87b3bbbfcdcdce33162ae79545e111e8a649bed953a4494613ef5ec9436e1a0b
3
+ size 5659083296
TableLLM-13b-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44ab26c4e0459dd0e2e6302284cd7e97b1fa98c3046429fce98c3e543b7c9778
3
+ size 7410185376
TableLLM-13b-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4cd1a88fe9367982f1ea1d2913a6ace450def8c9dcbee741f0776c52820e6840
3
+ size 7019629216
TableLLM-13b-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1336e7922a876451b5b3259a12a664cd29851a86a1de8be2eea780d4829e2918
3
+ size 4854364576
TableLLM-13b-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d58089f90c1b62c3c86784ffe28ec0869e2a9b84850d98fe1716f660c54f2c65
3
+ size 6929662496
TableLLM-13b-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6477f6a08225a7e34d70f73fdf64a5b3f3ba44f4ea869d0e90385f87914779e2
3
+ size 6337872416
TableLLM-13b-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:105ac9c36201e06cd8293f3f52a62671785c2d01cb3b04c63c848b0f282ad143
3
+ size 5659083296
TableLLM-13b-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d3bcc5b17038867e83f8b43fbc99bf95b4542cb30979eb1a33bf2cf88091d5cb
3
+ size 7365948576
TableLLM-13b-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cfa9e48c7a924da898aee15d85ec0903326c25f731a0a2a8c579d6b7caca955b
3
+ size 7866070176
TableLLM-13b-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b5e487412b3a3aaca7959f83cccd1fcdfbd49fc28d89f8200591596dfc3846a2
3
+ size 7423292576
TableLLM-13b-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d3a91336bc1e30e4cb2bc83b405c2b7d2af3f435c0cf6d255b4fbe55fdc2684d
3
+ size 8972410016
TableLLM-13b-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:438b49771570104394c12a21184274d0b55cfa161d60aa0d3334a8b102c27341
3
+ size 9230048416
TableLLM-13b-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:433208de69f32dd13e80621165930ad2c2623ff006b75f02e7c82b61a649c7df
3
+ size 8972410016
TableLLM-13b-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:811fb0c33330ef687b113fa58b65fa0e5b683c0297c7aa1c6ae44887cd150666
3
+ size 10679275296
TableLLM-13b-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:68a7c62090e1bbaf1d01e07ef8ad1e94b68dc750d7868598cabe31f12b67fec7
3
+ size 13831494176