justinthelaw
/

Phi-3-mini-128k-instruct-4bit-128g-GPTQ

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

justinthelaw commited on Aug 3

Commit

89a5fee

•

1 Parent(s): d93a852

Update README.md

Files changed (1) hide show

README.md +3 -2

README.md CHANGED Viewed

@@ -35,9 +35,10 @@ Models are released as sharded safetensors files.
 | Bits | GS | GPTQ Dataset | Max Seq Len | Size | VRAM |
 | ---- | -- | ----------- | ------- | ---- | ---- |
-| 4 | 128 | [wikitext2-v1](Salesforce/wikitext) | 131,072 | 2.53 Gb | 22-32 Gb*
 * Depends on maximum sequence length parameter (KV cache utilization) used with vLLM or Transformers
-*
 <!-- README_GPTQ.md-provided-files end -->
 ## Original Model Card Below

 | Bits | GS | GPTQ Dataset | Max Seq Len | Size | VRAM |
 | ---- | -- | ----------- | ------- | ---- | ---- |
+| 4 | 128 | [wikitext2-v1](Salesforce/wikitext) | 131,072 | 2.28 Gb | 22-32 Gb*
 * Depends on maximum sequence length parameter (KV cache utilization) used with vLLM or Transformers
 <!-- README_GPTQ.md-provided-files end -->
 ## Original Model Card Below