---
datasets:
- Marcus2112/minipile_density-proportioned
language:
- en
base_model:
- EleutherAI/pythia-1.4b-deduped
---

| Benchmark        | Measure    |     | 1.4B MiniPile              | 1.4B Density                   | Percentage Difference in Means |
| ---------------- | ---------- | --- | -------------------------- | ------------------------------ | ------------------------------ |
| ARC-Challenge    | acc        | ↑   | **0.1903 ± 0.0115**        | 0.1852 ± 0.0114                | -2.6800                        |
| MMLU             | acc        | ↑   | 0.2295 ± 0.0035            | 0.2295 ± 0.0035                | 0.0000                         |
| HellaSwag        | acc        | ↑   | 0.2579 ± 0.0044            | **0.2589 ± 0.0044**            | 0.3877                         |
| WinoGrande       | acc        | ↑   | **0.5185 ± 0.0140**        | 0.5043 ± 0.0141                | -2.7387                        |
| Lambada (OpenAI) | acc        | ↑   | 0.0000 ± 0.0000            | 0.0000 ± 0.0000                | -                              |
| Lambada (OpenAI) | perplexity | ↓   | 1564928.5258 ± 118691.4565 | **1420846.8323 ± 106563.1327** | -9.2069                        |
| Lambada (Std)    | acc        | ↑   | 0.0000 ± 0.0000            | 0.0000 ± 0.0000                | -                              |
| Lambada (Std)    | perplexity | ↓   | 8848600.9409 ± 745031.8900 | **7916035.3527 ± 664805.9178** | -10.5391                       |
| BLiMP            | acc        | ↑   | **0.5483 ± 0.0017**        | 0.5422 ± 0.0017                | -1.1125                        |
| ARC-Easy         | acc        | ↑   | **0.2715 ± 0.0091**        | 0.2698 ± 0.0091                | -0.6262                        |