Marcus2112's picture
Update README.md
e6c3269 verified
metadata
datasets:
  - Marcus2112/minipile_density-proportioned
language:
  - en
base_model:
  - EleutherAI/pythia-1.4b-deduped
Benchmark Measure 1.4B MiniPile 1.4B Density Percentage Difference in Means
ARC-Challenge acc 0.1903 ± 0.0115 0.1852 ± 0.0114 -2.6800
MMLU acc 0.2295 ± 0.0035 0.2295 ± 0.0035 0.0000
HellaSwag acc 0.2579 ± 0.0044 0.2589 ± 0.0044 0.3877
WinoGrande acc 0.5185 ± 0.0140 0.5043 ± 0.0141 -2.7387
Lambada (OpenAI) acc 0.0000 ± 0.0000 0.0000 ± 0.0000 -
Lambada (OpenAI) perplexity 1564928.5258 ± 118691.4565 1420846.8323 ± 106563.1327 -9.2069
Lambada (Std) acc 0.0000 ± 0.0000 0.0000 ± 0.0000 -
Lambada (Std) perplexity 8848600.9409 ± 745031.8900 7916035.3527 ± 664805.9178 -10.5391
BLiMP acc 0.5483 ± 0.0017 0.5422 ± 0.0017 -1.1125
ARC-Easy acc 0.2715 ± 0.0091 0.2698 ± 0.0091 -0.6262