Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
datasets:
|
| 3 |
+
- Marcus2112/minipile_density-proportioned
|
| 4 |
+
language:
|
| 5 |
+
- en
|
| 6 |
+
base_model:
|
| 7 |
+
- EleutherAI/pythia-160m-deduped
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
+
| Benchmark | Measure | | 160M Density | 160M Density 2 Epochs | Percentage Difference in Means |
|
| 11 |
+
| ---------------- | ---------- | --- | ---------------------------- | ------------------------------ | ------------------------------ |
|
| 12 |
+
| ARC-Challenge | acc | ↑ | **0.1920 ± 0.0115** | 0.1894 ± 0.0115 | -1.3542 |
|
| 13 |
+
| MMLU | acc | ↑ | 0.2295 ± 0.0035 | 0.2295 ± 0.0035 | 0.0000 |
|
| 14 |
+
| HellaSwag | acc | ↑ | **0.2604 ± 0.0044** | 0.2568 ± 0.0044 | -1.3825 |
|
| 15 |
+
| WinoGrande | acc | ↑ | **0.5201 ± 0.0140** | 0.5012 ± 0.0141 | -3.6339 |
|
| 16 |
+
| Lambada (OpenAI) | acc | ↑ | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | - |
|
| 17 |
+
| Lambada (OpenAI) | perplexity | ↓ | 2099002.0912 ± 170652.6222 | **1587737.3755 ± 121555.3148** | -24.3575 |
|
| 18 |
+
| Lambada (Std) | acc | ↑ | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | - |
|
| 19 |
+
| Lambada (Std) | perplexity | ↓ | 13347273.6076 ± 1997894.6360 | **8366924.7603 ± 713077.3579** | -37.3136 |
|
| 20 |
+
| BLiMP | acc | ↑ | **0.5501 ± 0.0017** | 0.5378 ± 0.0017 | -2.2360 |
|