MiniCorpus
Collection
github.com/MK2112/minicorpus
•
19 items
•
Updated
| Benchmark | Measure | 1.4B MiniPile | 1.4B Reproduction | Percentage Difference in Means | |
|---|---|---|---|---|---|
| ARC-Challenge | acc | ↑ | 0.1903 ± 0.0115 | 0.1928 ± 0.0115 | 1.3137 |
| MMLU | acc | ↑ | 0.2295 ± 0.0035 | 0.2295 ± 0.0035 | 0.0000 |
| HellaSwag | acc | ↑ | 0.2579 ± 0.0044 | 0.2584 ± 0.0044 | 0.1939 |
| WinoGrande | acc | ↑ | 0.5185 ± 0.0140 | 0.5091 ± 0.0141 | -1.8129 |
| Lambada (OpenAI) | acc | ↑ | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | - |
| Lambada (OpenAI) | perplexity | ↓ | 1564928.5258 ± 118691.4565 | 1520707.8702 ± 115261.3664 | -2.8257 |
| Lambada (Std) | acc | ↑ | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | - |
| Lambada (Std) | perplexity | ↓ | 8848600.9409 ± 745031.8900 | 8651201.8876 ± 735161.5236 | -1.5685 |
| BLiMP | acc | ↑ | 0.5483 ± 0.0017 | 0.5397 ± 0.0016 | -2.2309 |
| ARC-Easy | acc | ↑ | 0.2715 ± 0.0091 | 0.2673 ± 0.0091 | -1.5470 |
- Downloads last month
- 8
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.
Model tree for Marcus2112/pythia-1.4b-minipile_reproduction
Base model
EleutherAI/pythia-1.4b-deduped