Hugging Face's logo

csdc-atl
/
Baichuan2-7B-Chat-GPTQ-Int4

Text Generation
Transformers
Safetensors
English
Chinese
baichuan
custom_code
text-generation-inference
Inference Endpoints
4-bit precision
gptq
Model card Files Community
1
xiaotinghe commited on
Commit
c1e5aaf
•
1 Parent(s): 476c22a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -30,14 +30,14 @@ tasks:
30
  - GPTQ dataset: 中文、英文混合数据集
31
  - Sequence Length: 4096
32
 
33
- | 模型版本 | agieval | ceval | cmmlu | size | 推理速度 |
34
  |---|---|---|---|---|---|
35
- | [Baichuan2-7B-Chat](https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat) | 34.63 | 54.28 | 54.14 | 15g | |
36
- | [Baichuan2-7B-Chat-4bits](https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat-4bits) | 34.46 | 51.01 | 51.55 | 5.4g | |
37
- | [GPTQ-4bit-32g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/4bit-32g) | 34.11 | 52.74 | 52.92 | 5.8g | |
38
- | [GPTQ-4bit-128g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/main) | 33.42 | 52.17 | 52.05 | 5.4g | |
39
- | [GPTQ-8bit-32g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/8bit-32g) | | | | 9.2g | |
40
- | [GPTQ-8bit-128g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/8bit-128g) | | | | 8.7g | |
41
 
42
  <!-- README_GPTQ.md-provided-files end -->
43
  ## 如何在Python代码中使用此GPTQ模型
 
30
  - GPTQ dataset: 中文、英文混合数据集
31
  - Sequence Length: 4096
32
 
33
+ | 模型版本 | agieval | ceval | cmmlu | size | 推理速度(A100-40G) |
34
  |---|---|---|---|---|---|
35
+ | [Baichuan2-7B-Chat](https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat) | 34.63 | 54.28 | 54.14 | 15g | 30.87 tokens/s |
36
+ | [Baichuan2-7B-Chat-4bits](https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat-4bits) | 34.46 | 51.01 | 51.55 | 5.4g | 20.87 tokens/s |
37
+ | [GPTQ-4bit-32g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/4bit-32g) | 34.11 | 52.74 | 52.92 | 5.8g | 28.46 tokens/s |
38
+ | [GPTQ-4bit-128g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/main) | 33.42 | 52.17 | 52.05 | 5.4g | 28.39 tokens/s |
39
+ | [GPTQ-8bit-32g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/8bit-32g) | | | | 9.2g | 11.66 tokens/s |
40
+ | [GPTQ-8bit-128g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/8bit-128g) | | | | 8.7g | 12.83 tokens/s |
41
 
42
  <!-- README_GPTQ.md-provided-files end -->
43
  ## 如何在Python代码中使用此GPTQ模型