Vasily Alexeev commited on
Commit
79a3273
•
1 Parent(s): c27917d

add mmlu_other

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -35,10 +35,10 @@ Quantized with [OmniQuant](https://github.com/OpenGVLab/OmniQuant).
35
 
36
  ### Accuracy on English Benchmarks, % (↑)
37
 
38
- | | piqa | arc_easy | arc_challenge | boolq | hellaswag | winogrande | mmlu_humanities | mmlu_social_sciences | mmlu_stem |
39
- | ------------- | ---- | -------- | ------------- | ----- | --------- | ---------- | --------------- | -------------------- | --------- |
40
- | FP | 78,5 | 82,2 | 50,4 | 82,7 | 58,1 | 72,4 | 65,5 | 72,6 | 53,8 |
41
- | **Quantized** | 78,5 | 80,8 | 47,6 | 81,7 | 56,9 | 71,2 | 62,3 | 68,9 | 49,7 |
42
 
43
 
44
  ### Accuracy on Russian Benchmarks, % (↑)
 
35
 
36
  ### Accuracy on English Benchmarks, % (↑)
37
 
38
+ | | piqa | arc_easy | arc_challenge | boolq | hellaswag | winogrande | mmlu_humanities | mmlu_social_sciences | mmlu_stem | mmlu_other |
39
+ | ------------- | ---- | -------- | ------------- | ----- | --------- | ---------- | --------------- | -------------------- | --------- | ---------- |
40
+ | FP | 78,5 | 82,2 | 50,4 | 82,7 | 58,1 | 72,4 | 65,5 | 72,6 | 53,8 | 68,4 |
41
+ | **Quantized** | 78,5 | 80,8 | 47,6 | 81,7 | 56,9 | 71,2 | 62,3 | 68,9 | 49,7 | 63,3 |
42
 
43
 
44
  ### Accuracy on Russian Benchmarks, % (↑)