File size: 1,781 Bytes
7f9dd30 e6c3269 7f9dd30 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
---
datasets:
- Marcus2112/minipile_density-proportioned
language:
- en
base_model:
- EleutherAI/pythia-1.4b-deduped
---
| Benchmark | Measure | | 1.4B MiniPile | 1.4B Density | Percentage Difference in Means |
| ---------------- | ---------- | --- | -------------------------- | ------------------------------ | ------------------------------ |
| ARC-Challenge | acc | ↑ | **0.1903 ± 0.0115** | 0.1852 ± 0.0114 | -2.6800 |
| MMLU | acc | ↑ | 0.2295 ± 0.0035 | 0.2295 ± 0.0035 | 0.0000 |
| HellaSwag | acc | ↑ | 0.2579 ± 0.0044 | **0.2589 ± 0.0044** | 0.3877 |
| WinoGrande | acc | ↑ | **0.5185 ± 0.0140** | 0.5043 ± 0.0141 | -2.7387 |
| Lambada (OpenAI) | acc | ↑ | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | - |
| Lambada (OpenAI) | perplexity | ↓ | 1564928.5258 ± 118691.4565 | **1420846.8323 ± 106563.1327** | -9.2069 |
| Lambada (Std) | acc | ↑ | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | - |
| Lambada (Std) | perplexity | ↓ | 8848600.9409 ± 745031.8900 | **7916035.3527 ± 664805.9178** | -10.5391 |
| BLiMP | acc | ↑ | **0.5483 ± 0.0017** | 0.5422 ± 0.0017 | -1.1125 |
| ARC-Easy | acc | ↑ | **0.2715 ± 0.0091** | 0.2698 ± 0.0091 | -0.6262 | |