Marcus2112's picture
Update README.md
4135438 verified
metadata
datasets:
  - Marcus2112/minipile_density-proportioned_pico
language:
  - en
base_model:
  - EleutherAI/pythia-1.4b-deduped
Benchmark Measure 1.4B Density 1.4B Density Pico Percentage Difference in Means
ARC-Challenge acc 0.1852 ± 0.0114 0.1928 ± 0.0115 4.1037
MMLU acc 0.2295 ± 0.0035 0.2295 ± 0.0035 0.0000
HellaSwag acc 0.2589 ± 0.0044 0.2600 ± 0.0044 0.4249
WinoGrande acc 0.5043 ± 0.0141 0.5122 ± 0.0140 1.5665
Lambada (OpenAI) acc 0.0000 ± 0.0000 0.0000 ± 0.0000 -
Lambada (OpenAI) perplexity 1420846.8323 ± 106563.1327 1662608.9444 ± 128444.3607 17.0154
Lambada (Std) acc 0.0000 ± 0.0000 0.0000 ± 0.0000 -
Lambada (Std) perplexity 7916035.3527 ± 664805.9178 8543578.1832 ± 737889.9436 7.9654
BLiMP acc 0.5422 ± 0.0017 0.5445 ± 0.0017 7.9654
ARC-Easy acc 0.2698 ± 0.0091 0.2761 ± 0.0091 2.3351