hivata
/

functionary-small-v3.2-AWQ

4-bit precision

Model card Files Files and versions Community

LHC88 commited on Aug 9, 2024

Commit

e8232cf

·

verified ·

1 Parent(s): 374e22b

Update README.md

Files changed (1) hide show

README.md +8 -8

README.md CHANGED Viewed

@@ -7,14 +7,14 @@ Available AWQ Quantisations:
 |   Revision   | Model Size | Group Size | w_bit |
 |--------------|------------|------------|-------|
-|    `main`    |   ~6.9 GB  |     32     |   4   |
-|  `gs_2_4bit` |   ~6.9 GB  |      2     |   4   |
-|  `gs_4_4bit` |   ~6.9 GB  |      4     |   4   |
-|  `gs_8_4bit` |   ~6.9 GB  |      8     |   4   |
-| `gs_16_4bit` |   ~6.9 GB  |     16     |   4   |
-| `gs_64_4bit` |   ~6.9 GB  |     64     |   4   |
-| `gs_128_4bit`|   ~6.9 GB  |    128     |   4   |
-| `gs_512_4bit`|   ~6.9 GB  |    512     |   4   |
 **This model was based on [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)**

 |   Revision   | Model Size | Group Size | w_bit |
 |--------------|------------|------------|-------|
+|    `main`    |  ~6.13 GB  |     32     |   4   |
+|  `gs_2_4bit` |  ~x.xx GB  |      2     |   4   |
+|  `gs_4_4bit` |  ~x.xx GB  |      4     |   4   |
+|  `gs_8_4bit` |  ~x.xx GB  |      8     |   4   |
+| [`gs_16_4bit`](https://huggingface.co/hivata/functionary-small-v3.2-AWQ/tree/gs_16-4bit) |  ~6.68 GB  |     16     |   4   |
+| `gs_64_4bit` |  ~x.xx GB  |     64     |   4   |
+| `gs_128_4bit`|  ~x.xx GB  |    128     |   4   |
+| `gs_512_4bit`|  ~x.xx GB  |    512     |   4   |
 **This model was based on [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)**