Vasily Alexeev
commited on
Commit
·
79a3273
1
Parent(s):
c27917d
add mmlu_other
Browse files
README.md
CHANGED
@@ -35,10 +35,10 @@ Quantized with [OmniQuant](https://github.com/OpenGVLab/OmniQuant).
|
|
35 |
|
36 |
### Accuracy on English Benchmarks, % (↑)
|
37 |
|
38 |
-
| | piqa | arc_easy | arc_challenge | boolq | hellaswag | winogrande | mmlu_humanities | mmlu_social_sciences | mmlu_stem |
|
39 |
-
| ------------- | ---- | -------- | ------------- | ----- | --------- | ---------- | --------------- | -------------------- | --------- |
|
40 |
-
| FP | 78,5 | 82,2 | 50,4 | 82,7 | 58,1 | 72,4 | 65,5 | 72,6 | 53,8 |
|
41 |
-
| **Quantized** | 78,5 | 80,8 | 47,6 | 81,7 | 56,9 | 71,2 | 62,3 | 68,9 | 49,7 |
|
42 |
|
43 |
|
44 |
### Accuracy on Russian Benchmarks, % (↑)
|
|
|
35 |
|
36 |
### Accuracy on English Benchmarks, % (↑)
|
37 |
|
38 |
+
| | piqa | arc_easy | arc_challenge | boolq | hellaswag | winogrande | mmlu_humanities | mmlu_social_sciences | mmlu_stem | mmlu_other |
|
39 |
+
| ------------- | ---- | -------- | ------------- | ----- | --------- | ---------- | --------------- | -------------------- | --------- | ---------- |
|
40 |
+
| FP | 78,5 | 82,2 | 50,4 | 82,7 | 58,1 | 72,4 | 65,5 | 72,6 | 53,8 | 68,4 |
|
41 |
+
| **Quantized** | 78,5 | 80,8 | 47,6 | 81,7 | 56,9 | 71,2 | 62,3 | 68,9 | 49,7 | 63,3 |
|
42 |
|
43 |
|
44 |
### Accuracy on Russian Benchmarks, % (↑)
|