T145 commited on
Commit
a0df6ed
·
verified ·
1 Parent(s): d40c15b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -2
README.md CHANGED
@@ -104,9 +104,9 @@ model-index:
104
  url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=T145/ZEUS-8B-V10
105
  name: Open LLM Leaderboard
106
  ---
107
- # ZEUS 8B V10
108
 
109
- A simple V2 recreation with a few changes:
110
 
111
  * Unified tokenizer (no noticeable changes)
112
  * Using `int_mask` and `normalize` (the latter being enabled by default in mergekit)
@@ -173,3 +173,14 @@ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-le
173
  |MuSR (0-shot) | 9.09|
174
  |MMLU-PRO (5-shot) |32.26|
175
 
 
 
 
 
 
 
 
 
 
 
 
 
104
  url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=T145/ZEUS-8B-V10
105
  name: Open LLM Leaderboard
106
  ---
107
+ # ZEUS 8B 🌩️ V10
108
 
109
+ A V2 recreation with a few changes:
110
 
111
  * Unified tokenizer (no noticeable changes)
112
  * Using `int_mask` and `normalize` (the latter being enabled by default in mergekit)
 
173
  |MuSR (0-shot) | 9.09|
174
  |MMLU-PRO (5-shot) |32.26|
175
 
176
+ ## Changes over V2
177
+
178
+ | Metric |Change|
179
+ |-------------------|-----:|
180
+ |Avg. |+0.12|
181
+ |IFEval (0-Shot) |-3.22|
182
+ |BBH (3-Shot) |+1.09|
183
+ |MATH Lvl 5 (4-Shot)|-1.06|
184
+ |GPQA (0-shot) |+3.02|
185
+ |MuSR (0-shot) |+0.85|
186
+ |MMLU-PRO (5-shot) |+0.08|