Hugging Face's logo

mradermacher
/
DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF

Transformers
GGUF
English
Llama-3
instruct
finetune
chatml
gpt4
synthetic data
distillation
function calling
json mode
axolotl
roleplaying
chat
reasoning
r1
vllm
abliterated
uncensored
Inference Endpoints
conversational
Model card Files Community
mradermacher commited on
Commit
183552d
·
verified ·
1 Parent(s): 5e5f33f

auto-patch README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -1
README.md CHANGED
@@ -34,7 +34,7 @@ tags:
34
  static quants of https://huggingface.co/huihui-ai/DeepHermes-3-Llama-3-8B-Preview-abliterated
35
 
36
  <!-- provided-files -->
37
- weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion.
38
  ## Usage
39
 
40
  If you are unsure how to use GGUF files, refer to one of [TheBloke's
@@ -47,6 +47,17 @@ more details, including on how to concatenate multi-part files.
47
 
48
  | Link | Type | Size/GB | Notes |
49
  |:-----|:-----|--------:|:------|
 
 
 
 
 
 
 
 
 
 
 
50
  | [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF/resolve/main/DeepHermes-3-Llama-3-8B-Preview-abliterated.f16.gguf) | f16 | 16.2 | 16 bpw, overkill |
51
 
52
  Here is a handy graph by ikawrakow comparing some lower-quality quant
 
34
  static quants of https://huggingface.co/huihui-ai/DeepHermes-3-Llama-3-8B-Preview-abliterated
35
 
36
  <!-- provided-files -->
37
+ weighted/imatrix quants are available at https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-i1-GGUF
38
  ## Usage
39
 
40
  If you are unsure how to use GGUF files, refer to one of [TheBloke's
 
47
 
48
  | Link | Type | Size/GB | Notes |
49
  |:-----|:-----|--------:|:------|
50
+ | [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF/resolve/main/DeepHermes-3-Llama-3-8B-Preview-abliterated.Q2_K.gguf) | Q2_K | 3.3 | |
51
+ | [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF/resolve/main/DeepHermes-3-Llama-3-8B-Preview-abliterated.Q3_K_S.gguf) | Q3_K_S | 3.8 | |
52
+ | [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF/resolve/main/DeepHermes-3-Llama-3-8B-Preview-abliterated.Q3_K_M.gguf) | Q3_K_M | 4.1 | lower quality |
53
+ | [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF/resolve/main/DeepHermes-3-Llama-3-8B-Preview-abliterated.Q3_K_L.gguf) | Q3_K_L | 4.4 | |
54
+ | [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF/resolve/main/DeepHermes-3-Llama-3-8B-Preview-abliterated.IQ4_XS.gguf) | IQ4_XS | 4.6 | |
55
+ | [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF/resolve/main/DeepHermes-3-Llama-3-8B-Preview-abliterated.Q4_K_S.gguf) | Q4_K_S | 4.8 | fast, recommended |
56
+ | [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF/resolve/main/DeepHermes-3-Llama-3-8B-Preview-abliterated.Q4_K_M.gguf) | Q4_K_M | 5.0 | fast, recommended |
57
+ | [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF/resolve/main/DeepHermes-3-Llama-3-8B-Preview-abliterated.Q5_K_S.gguf) | Q5_K_S | 5.7 | |
58
+ | [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF/resolve/main/DeepHermes-3-Llama-3-8B-Preview-abliterated.Q5_K_M.gguf) | Q5_K_M | 5.8 | |
59
+ | [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF/resolve/main/DeepHermes-3-Llama-3-8B-Preview-abliterated.Q6_K.gguf) | Q6_K | 6.7 | very good quality |
60
+ | [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF/resolve/main/DeepHermes-3-Llama-3-8B-Preview-abliterated.Q8_0.gguf) | Q8_0 | 8.6 | fast, best quality |
61
  | [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Llama-3-8B-Preview-abliterated-GGUF/resolve/main/DeepHermes-3-Llama-3-8B-Preview-abliterated.f16.gguf) | f16 | 16.2 | 16 bpw, overkill |
62
 
63
  Here is a handy graph by ikawrakow comparing some lower-quality quant