H2O Danube3
Collection
7 items
•
Updated
•
55
This repo contains GGUF format model files for h2o-danube3-500m-chat quantized using llama.cpp framework.
Table below summarizes different quantized versions of h2o-danube3-500m-chat. It shows the trade-off between size, speed and quality of the models.
Name | Quant method | Model size | MT-Bench AVG | Perplexity | Tokens per second |
---|---|---|---|---|---|
h2o-danube3-500m-chat-F16.gguf | F16 | 1.03 GB | 3.34 | 9.46 | 1870 |
h2o-danube3-500m-chat-Q8_0.gguf | Q8_0 | 0.55 GB | 3.76 | 9.46 | 2144 |
h2o-danube3-500m-chat-Q6_K.gguf | Q6_K | 0.42 GB | 3.77 | 9.46 | 2418 |
h2o-danube3-500m-chat-Q5_K_M.gguf | Q5_K_M | 0.37 GB | 3.20 | 9.55 | 2430 |
h2o-danube3-500m-chat-Q4_K_M.gguf | Q4_K_M | 0.32 GB | 3.16 | 9.96 | 2427 |
Columns in the table are:
<|prompt|>Why is drinking water so healthy?</s><|answer|>