Llamacpp Quantizations of AlphaHitchhiker-7B-v2

Using llama.cpp release b2405 for quantization.

Original model: https://huggingface.co/macadeliccc/AlphaHitchhiker-7B-v2

Download a file (not the whole branch) from below:

Filename Quant type File Size Description
AlphaHitchhiker-7B-v2-Q8_0.gguf Q8_0 7.69GB Extremely high quality, generally unneeded but max available quant.
AlphaHitchhiker-7B-v2-Q6_K.gguf Q6_K 5.94GB Very high quality, near perfect, recommended.
AlphaHitchhiker-7B-v2-Q5_K_M.gguf Q5_K_M 5.13GB High quality, very usable.
AlphaHitchhiker-7B-v2-Q5_K_S.gguf Q5_K_S 4.99GB High quality, very usable.
AlphaHitchhiker-7B-v2-Q5_0.gguf Q5_0 4.99GB High quality, older format, generally not recommended.
AlphaHitchhiker-7B-v2-Q4_K_M.gguf Q4_K_M 4.36GB Good quality, similar to 4.25 bpw.
AlphaHitchhiker-7B-v2-Q4_K_S.gguf Q4_K_S 4.14GB Slightly lower quality with small space savings.
AlphaHitchhiker-7B-v2-Q4_0.gguf Q4_0 4.10GB Decent quality, older format, generally not recommended.
AlphaHitchhiker-7B-v2-Q3_K_L.gguf Q3_K_L 3.82GB Lower quality but usable, good for low RAM availability.
AlphaHitchhiker-7B-v2-Q3_K_M.gguf Q3_K_M 3.51GB Even lower quality.
AlphaHitchhiker-7B-v2-Q3_K_S.gguf Q3_K_S 3.16GB Low quality, not recommended.
AlphaHitchhiker-7B-v2-Q2_K.gguf Q2_K 2.71GB Extremely low quality, not recommended.

Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

Downloads last month
101
GGUF
Model size
7.24B params
Architecture
llama

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for bartowski/AlphaHitchhiker-7B-v2-GGUF

Quantized
(9)
this model