fedric95
/

gemma-7b-GGUF

Text Generation

Inference Endpoints

Model card Files Files and versions Community

fedric95 commited on Aug 22, 2024

Commit

aa98108

·

verified ·

1 Parent(s): cbe497a

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -23,7 +23,7 @@ Original model: https://huggingface.co/google/gemma-7b
 | Filename | Quant type | File Size | Perplexity (wikitext-2-raw-v1.test) |
 | -------- | ---------- | --------- | ----------- |
-| [gemma-7b.FP32.gguf](https://huggingface.co/fedric95/gemma-7b-GGUF/blob/main/gemma-7b.FP32.gguf) | BF16 | 17.1 GB | 6.9857 +/- 0.04411 |
 | [gemma-7b-Q8_0.gguf](https://huggingface.co/fedric95/gemma-7b-GGUF/blob/main/gemma-7b-Q8_0.gguf) | Q8_0 | 9.08 GB | 7.0373 +/- 0.04456 |
 | [gemma-7b-Q6_K.gguf](https://huggingface.co/fedric95/gemma-7b-GGUF/blob/main/gemma-7b-Q6_K.gguf) | Q6_K | 7.01 GB | 7.3858 +/- 0.04762 |
 | [gemma-7b-Q5_K_M.gguf](https://huggingface.co/fedric95/gemma-7b-GGUF/blob/main/gemma-7b-Q5_K_M.gguf) | Q5_K_M | 6.14 GB | 7.4227 +/- 0.04781 |

 | Filename | Quant type | File Size | Perplexity (wikitext-2-raw-v1.test) |
 | -------- | ---------- | --------- | ----------- |
+| [gemma-7b.BF16.gguf](https://huggingface.co/fedric95/gemma-7b-GGUF/blob/main/gemma-7b.BF16.gguf) | BF16 | 17.1 GB | 6.9857 +/- 0.04411 |
 | [gemma-7b-Q8_0.gguf](https://huggingface.co/fedric95/gemma-7b-GGUF/blob/main/gemma-7b-Q8_0.gguf) | Q8_0 | 9.08 GB | 7.0373 +/- 0.04456 |
 | [gemma-7b-Q6_K.gguf](https://huggingface.co/fedric95/gemma-7b-GGUF/blob/main/gemma-7b-Q6_K.gguf) | Q6_K | 7.01 GB | 7.3858 +/- 0.04762 |
 | [gemma-7b-Q5_K_M.gguf](https://huggingface.co/fedric95/gemma-7b-GGUF/blob/main/gemma-7b-Q5_K_M.gguf) | Q5_K_M | 6.14 GB | 7.4227 +/- 0.04781 |