Hugging Face's logo

csdc-atl
/
Baichuan2-7B-Chat-GPTQ-Int4

Text Generation
Transformers
Safetensors
English
Chinese
baichuan
custom_code
text-generation-inference
Inference Endpoints
4-bit precision
gptq
Model card Files Community
1
xiaotinghe commited on
Commit
3903d0d
•
1 Parent(s): c1e5aaf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -34,8 +34,8 @@ tasks:
34
  |---|---|---|---|---|---|
35
  | [Baichuan2-7B-Chat](https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat) | 34.63 | 54.28 | 54.14 | 15g | 30.87 tokens/s |
36
  | [Baichuan2-7B-Chat-4bits](https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat-4bits) | 34.46 | 51.01 | 51.55 | 5.4g | 20.87 tokens/s |
37
- | [GPTQ-4bit-32g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/4bit-32g) | 34.11 | 52.74 | 52.92 | 5.8g | 28.46 tokens/s |
38
- | [GPTQ-4bit-128g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/main) | 33.42 | 52.17 | 52.05 | 5.4g | 28.39 tokens/s |
39
  | [GPTQ-8bit-32g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/8bit-32g) | | | | 9.2g | 11.66 tokens/s |
40
  | [GPTQ-8bit-128g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/8bit-128g) | | | | 8.7g | 12.83 tokens/s |
41
 
 
34
  |---|---|---|---|---|---|
35
  | [Baichuan2-7B-Chat](https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat) | 34.63 | 54.28 | 54.14 | 15g | 30.87 tokens/s |
36
  | [Baichuan2-7B-Chat-4bits](https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat-4bits) | 34.46 | 51.01 | 51.55 | 5.4g | 20.87 tokens/s |
37
+ | [GPTQ-4bit-32g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/4bit-32g) | 34.11 | 52.74 | 52.92 | 5.8g | 28.46(hf) \ 37.77(autogptq) tokens/s |
38
+ | [GPTQ-4bit-128g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/main) | 33.42 | 52.17 | 52.05 | 5.4g | 28.39(hf) \ 37.53(autogptq) tokens/s |
39
  | [GPTQ-8bit-32g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/8bit-32g) | | | | 9.2g | 11.66 tokens/s |
40
  | [GPTQ-8bit-128g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/8bit-128g) | | | | 8.7g | 12.83 tokens/s |
41