Update README.md
Browse files
README.md
CHANGED
@@ -104,9 +104,9 @@ model-index:
|
|
104 |
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=T145/ZEUS-8B-V10
|
105 |
name: Open LLM Leaderboard
|
106 |
---
|
107 |
-
# ZEUS 8B V10
|
108 |
|
109 |
-
A
|
110 |
|
111 |
* Unified tokenizer (no noticeable changes)
|
112 |
* Using `int_mask` and `normalize` (the latter being enabled by default in mergekit)
|
@@ -173,3 +173,14 @@ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-le
|
|
173 |
|MuSR (0-shot) | 9.09|
|
174 |
|MMLU-PRO (5-shot) |32.26|
|
175 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
104 |
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=T145/ZEUS-8B-V10
|
105 |
name: Open LLM Leaderboard
|
106 |
---
|
107 |
+
# ZEUS 8B 🌩️ V10
|
108 |
|
109 |
+
A V2 recreation with a few changes:
|
110 |
|
111 |
* Unified tokenizer (no noticeable changes)
|
112 |
* Using `int_mask` and `normalize` (the latter being enabled by default in mergekit)
|
|
|
173 |
|MuSR (0-shot) | 9.09|
|
174 |
|MMLU-PRO (5-shot) |32.26|
|
175 |
|
176 |
+
## Changes over V2
|
177 |
+
|
178 |
+
| Metric |Change|
|
179 |
+
|-------------------|-----:|
|
180 |
+
|Avg. |+0.12|
|
181 |
+
|IFEval (0-Shot) |-3.22|
|
182 |
+
|BBH (3-Shot) |+1.09|
|
183 |
+
|MATH Lvl 5 (4-Shot)|-1.06|
|
184 |
+
|GPQA (0-shot) |+3.02|
|
185 |
+
|MuSR (0-shot) |+0.85|
|
186 |
+
|MMLU-PRO (5-shot) |+0.08|
|