Hugging Face's logo

DevsDoCode
/
LLama-3-8b-Uncensored-Q2_K-GGUF

Transformers
GGUF
English
uncensored
llama
llama-3
unsloth
llama-factory
Inference Endpoints
conversational
Model card Files Community
DevsDoCode commited on
Commit
1f31073
·
verified ·
1 Parent(s): f63fa6f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +72 -23
README.md CHANGED
@@ -1,36 +1,85 @@
1
  ---
 
 
 
2
  library_name: transformers
 
 
3
  tags:
4
- - llama-cpp
5
- - gguf-my-repo
 
 
 
 
6
  ---
 
 
 
 
 
 
 
 
7
 
8
- # DevsDoCode/LLama-3-8b-Uncensored-Q2_K-GGUF
9
- This model was converted to GGUF format from [`DevsDoCode/LLama-3-8b-Uncensored`](https://huggingface.co/DevsDoCode/LLama-3-8b-Uncensored) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
10
- Refer to the [original model card](https://huggingface.co/DevsDoCode/LLama-3-8b-Uncensored) for more details on the model.
11
- ## Use with llama.cpp
12
 
13
- Install llama.cpp through brew.
 
14
 
15
- ```bash
16
- brew install ggerganov/ggerganov/llama.cpp
17
- ```
18
- Invoke the llama.cpp server or the CLI.
 
19
 
20
- CLI:
 
 
21
 
22
- ```bash
23
- llama-cli --hf-repo DevsDoCode/LLama-3-8b-Uncensored-Q2_K-GGUF --model llama-3-8b-uncensored.Q2_K.gguf -p "The meaning to life and the universe is"
24
- ```
25
 
26
- Server:
27
 
28
- ```bash
29
- llama-server --hf-repo DevsDoCode/LLama-3-8b-Uncensored-Q2_K-GGUF --model llama-3-8b-uncensored.Q2_K.gguf -c 2048
30
- ```
31
 
32
- Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
 
34
- ```
35
- git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make && ./main -m llama-3-8b-uncensored.Q2_K.gguf -n 128
36
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ base_model: DevsDoCode/LLama-3-8b-Uncensored
3
+ language:
4
+ - en
5
  library_name: transformers
6
+ license: apache-2.0
7
+ quantized_by: mradermacher
8
  tags:
9
+ - uncensored
10
+ - transformers
11
+ - llama
12
+ - llama-3
13
+ - unsloth
14
+ - llama-factory
15
  ---
16
+ <div align="center">
17
+ <!-- Replace `#` with your actual links -->
18
+ <a href="https://youtube.com/@devsdocode"><img alt="YouTube" src="https://img.shields.io/badge/YouTube-FF0000?style=for-the-badge&logo=youtube&logoColor=white"></a>
19
+ <a href="https://t.me/devsdocode"><img alt="Telegram" src="https://img.shields.io/badge/Telegram-2CA5E0?style=for-the-badge&logo=telegram&logoColor=white"></a>
20
+ <a href="https://www.instagram.com/sree.shades_/"><img alt="Instagram" src="https://img.shields.io/badge/Instagram-E4405F?style=for-the-badge&logo=instagram&logoColor=white"></a>
21
+ <a href="https://www.linkedin.com/in/developer-sreejan/"><img alt="LinkedIn" src="https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white"></a>
22
+ <a href="https://buymeacoffee.com/devsdocode"><img alt="Buy Me A Coffee" src="https://img.shields.io/badge/Buy%20Me%20A%20Coffee-FFDD00?style=for-the-badge&logo=buymeacoffee&logoColor=black"></a>
23
+ </div>
24
 
 
 
 
 
25
 
26
+ ## Crafted with ❤️ by Devs Do Code (Sree)
27
+ ## About
28
 
29
+ <!-- ### quantize_version: 1 -->
30
+ <!-- ### output_tensor_quantised: 1 -->
31
+ <!-- ### convert_type: -->
32
+ <!-- ### vocab_type: -->
33
+ static quants of https://huggingface.co/DevsDoCode/LLama-3-8b-Uncensored
34
 
35
+ <!-- provided-files -->
36
+ weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion.
37
+ ## Usage
38
 
39
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
40
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
41
+ more details, including on how to concatenate multi-part files.
42
 
43
+ ## Provided Quants
44
 
45
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
 
 
46
 
47
+ | Link | Type | Size/GB | Notes |
48
+ |:-----|:-----|--------:|:------|
49
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.Q2_K.gguf) | Q2_K | 3.3 | |
50
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.IQ3_XS.gguf) | IQ3_XS | 3.6 | |
51
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.Q3_K_S.gguf) | Q3_K_S | 3.8 | |
52
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.IQ3_S.gguf) | IQ3_S | 3.8 | beats Q3_K* |
53
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.IQ3_M.gguf) | IQ3_M | 3.9 | |
54
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.Q3_K_M.gguf) | Q3_K_M | 4.1 | lower quality |
55
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.Q3_K_L.gguf) | Q3_K_L | 4.4 | |
56
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.IQ4_XS.gguf) | IQ4_XS | 4.6 | |
57
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.Q4_K_S.gguf) | Q4_K_S | 4.8 | fast, recommended |
58
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.Q4_K_M.gguf) | Q4_K_M | 5.0 | fast, recommended |
59
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.Q5_K_S.gguf) | Q5_K_S | 5.7 | |
60
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.Q5_K_M.gguf) | Q5_K_M | 5.8 | |
61
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.Q6_K.gguf) | Q6_K | 6.7 | very good quality |
62
+ | [GGUF](https://huggingface.co/mradermacher/DevsDoCode-LLama-3-8b-Uncensored-GGUF/resolve/main/DevsDoCode-LLama-3-8b-Uncensored.Q8_0.gguf) | Q8_0 | 8.6 | fast, best quality |
63
 
64
+
65
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
66
+ types (lower is better):
67
+
68
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
69
+
70
+ And here are Artefact2's thoughts on the matter:
71
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
72
+
73
+ ## FAQ / Model Request
74
+
75
+ See https://huggingface.co/mradermacher/model_requests for some answers to
76
+ questions you might have and/or if you want some other model quantized.
77
+
78
+ <div align="center">
79
+ <!-- Replace `#` with your actual links -->
80
+ <a href="https://youtube.com/@devsdocode"><img alt="YouTube" src="https://img.shields.io/badge/YouTube-FF0000?style=for-the-badge&logo=youtube&logoColor=white"></a>
81
+ <a href="https://t.me/devsdocode"><img alt="Telegram" src="https://img.shields.io/badge/Telegram-2CA5E0?style=for-the-badge&logo=telegram&logoColor=white"></a>
82
+ <a href="https://www.instagram.com/sree.shades_/"><img alt="Instagram" src="https://img.shields.io/badge/Instagram-E4405F?style=for-the-badge&logo=instagram&logoColor=white"></a>
83
+ <a href="https://www.linkedin.com/in/developer-sreejan/"><img alt="LinkedIn" src="https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white"></a>
84
+ <a href="https://buymeacoffee.com/devsdocode"><img alt="Buy Me A Coffee" src="https://img.shields.io/badge/Buy%20Me%20A%20Coffee-FFDD00?style=for-the-badge&logo=buymeacoffee&logoColor=black"></a>
85
+ </div>