lapp0
/

distily_bench_gpt2_optim_extended2

@@ -16,13 +16,13 @@ This student model is distilled from the teacher model [gpt2](https://huggingfac
 The [Distily](https://github.com/lapp0/distily) library was used for this distillation.
 It achieves the following results on the evaluation set:
-- eval_enwikippl: 859.1765
-- eval_frwikippl: 4914.2627
-- eval_zhwikippl: 7375.7471
-- eval_loss: 6893.8027
-- eval_runtime: 65.3454
-- eval_samples_per_second: 45.91
-- eval_steps_per_second: 11.477
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment.
@@ -59,51 +59,51 @@ The following hyperparameters were used during training:
 - num_epochs: 1.0
 ### Resource Usage
-Peak GPU Memory: 8.3331 GB
 ### Eval-Phase Metrics
 | step | epoch | enwikippl | frwikippl | loss | runtime | samples_per_second | steps_per_second | zhwikippl |
 | --- | --- | --- | --- | --- | --- | --- | --- | --- |
 | **teacher eval** |  | 30.2385 | 57.2728 |  |  |  |  | 18.1772 |
-| 0 | 0 | 59513.7266 | 59655.5039 | 330435.25 | 63.4949 | 47.248 | 11.812 | 53382.8789 |
-| 500 | 0.0269 | 2863.2634 | 11508.8330 | 12259.9258 | 63.8689 | 46.971 | 11.743 | 38858.9141 |
-| 1000 | 0.0539 | 2136.9172 | 7522.9854 | 10143.5098 | 63.9165 | 46.936 | 11.734 | 16435.1426 |
-| 1500 | 0.0808 | 1930.7159 | 7313.2690 | 9805.8242 | 64.0662 | 46.827 | 11.707 | 14467.3398 |
-| 2000 | 0.1077 | 1691.7502 | 6743.2197 | 9288.6826 | 64.7936 | 46.301 | 11.575 | 13923.3652 |
-| 2500 | 0.1347 | 1564.1060 | 6471.2065 | 8980.0957 | 65.967 | 45.477 | 11.369 | 13381.9658 |
-| 3000 | 0.1616 | 1452.3389 | 6264.2476 | 8701.7598 | 65.1396 | 46.055 | 11.514 | 14317.4297 |
-| 3500 | 0.1886 | 1346.0464 | 6201.8452 | 8515.4990 | 64.711 | 46.36 | 11.59 | 14058.8223 |
-| 4000 | 0.2155 | 1293.1278 | 5995.4678 | 8215.7871 | 64.881 | 46.238 | 11.56 | 14290.6924 |
-| 4500 | 0.2424 | 1188.8944 | 5653.4883 | 7998.4854 | 64.8042 | 46.293 | 11.573 | 12600.6602 |
-| 5000 | 0.2694 | 1150.8783 | 5734.7847 | 7816.5864 | 65.0575 | 46.113 | 11.528 | 11801.7627 |
-| 5500 | 0.2963 | 1106.1051 | 5579.8325 | 7761.1094 | 64.8301 | 46.275 | 11.569 | 10491.8916 |
-| 6000 | 0.3232 | 1062.4747 | 5325.9385 | 7568.8638 | 64.8331 | 46.273 | 11.568 | 9862.1807 |
-| 6500 | 0.3502 | 1016.8871 | 5219.4351 | 7424.0107 | 65.1855 | 46.023 | 11.506 | 8266.7178 |
-| 7000 | 0.3771 | 990.4482 | 5391.6797 | 7382.8267 | 64.9653 | 46.179 | 11.545 | 8352.1592 |
-| 7500 | 0.4040 | 957.9040 | 5171.6309 | 7296.6401 | 64.9498 | 46.19 | 11.547 | 7235.2671 |
-| 8000 | 0.4310 | 936.6096 | 5295.7979 | 7155.7334 | 64.9012 | 46.224 | 11.556 | 8302.1201 |
-| 8500 | 0.4579 | 893.5040 | 5091.3115 | 7057.3013 | 65.0288 | 46.133 | 11.533 | 7185.6812 |
-| 9000 | 0.4848 | 894.5970 | 5188.4321 | 6994.9229 | 65.1952 | 46.016 | 11.504 | 8320.9854 |
-| 9500 | 0.5118 | 859.1765 | 4914.2627 | 6893.8027 | 65.3454 | 45.91 | 11.477 | 7375.7471 |
-| 10000 | 0.5387 | 844.4594 | 5006.5923 | 6830.0801 | 64.9208 | 46.21 | 11.553 | 7772.1235 |
-| 10500 | 0.5657 | 825.0941 | 4731.5112 | 6761.4507 | 64.7216 | 46.352 | 11.588 | 6593.5493 |
-| 11000 | 0.5926 | 805.9855 | 4779.2881 | 6747.7651 | 65.2107 | 46.005 | 11.501 | 6463.2256 |
-| 11500 | 0.6195 | 798.3549 | 4823.9761 | 6676.7681 | 65.0141 | 46.144 | 11.536 | 9802.4414 |
-| 12000 | 0.6465 | 791.6104 | 4696.2798 | 6579.1147 | 64.8969 | 46.227 | 11.557 | 7084.2090 |
-| 12500 | 0.6734 | 761.6416 | 4785.1895 | 6548.4907 | 64.8647 | 46.25 | 11.563 | 9604.8457 |
-| 13000 | 0.7003 | 761.1536 | 4701.0835 | 6501.8882 | 65.0795 | 46.097 | 11.524 | 6464.0889 |
-| 13500 | 0.7273 | 734.9013 | 4585.2031 | 6514.5386 | 65.3072 | 45.937 | 11.484 | 6561.9272 |
-| 14000 | 0.7542 | 740.7595 | 4580.6797 | 6382.9121 | 65.4037 | 45.869 | 11.467 | 6729.2021 |
-| 14500 | 0.7811 | 711.6922 | 4399.3062 | 6421.3013 | 64.9755 | 46.171 | 11.543 | 10408.1582 |
-| 15000 | 0.8081 | 699.6238 | 4419.9839 | 6434.4214 | 64.9281 | 46.205 | 11.551 | 6650.1450 |
-| 15500 | 0.8350 | 689.9260 | 4160.6733 | 6344.1709 | 65.0911 | 46.089 | 11.522 | 5445.5518 |
-| 16000 | 0.8620 | 690.0198 | 4297.5234 | 6258.8799 | 64.8144 | 46.286 | 11.572 | 4706.0220 |
-| 16500 | 0.8889 | 691.7233 | 4264.9258 | 6328.2451 | 65.5974 | 45.734 | 11.433 | 5371.5254 |
-| 17000 | 0.9158 | 665.1479 | 4280.1362 | 6238.0371 | 64.909 | 46.219 | 11.555 | 7401.3955 |
-| 17500 | 0.9428 | 668.0211 | 4168.7480 | 6233.2695 | 64.7669 | 46.32 | 11.58 | 6142.2017 |
-| 18000 | 0.9697 | 653.9958 | 4099.5210 | 6188.6719 | 64.1993 | 46.729 | 11.682 | 5851.9600 |
-| 18500 | 0.9966 | 644.6927 | 3924.1768 | 6175.1572 | 64.4251 | 46.566 | 11.641 | 5162.6572 |
-| 18562 | 1.0000 | 649.2895 | 3895.5068 | 6124.0107 | 64.4839 | 46.523 | 11.631 | 4821.7959 |
 ### Framework versions
 - Distily 0.2.0

 The [Distily](https://github.com/lapp0/distily) library was used for this distillation.
 It achieves the following results on the evaluation set:
+- eval_enwikippl: 603.2673
+- eval_frwikippl: 3866.3679
+- eval_zhwikippl: 9060.9883
+- eval_loss: 6355.0508
+- eval_runtime: 64.6366
+- eval_samples_per_second: 46.413
+- eval_steps_per_second: 11.603
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment.
 - num_epochs: 1.0
 ### Resource Usage
+Peak GPU Memory: 8.3344 GB
 ### Eval-Phase Metrics
 | step | epoch | enwikippl | frwikippl | loss | runtime | samples_per_second | steps_per_second | zhwikippl |
 | --- | --- | --- | --- | --- | --- | --- | --- | --- |
 | **teacher eval** |  | 30.2385 | 57.2728 |  |  |  |  | 18.1772 |
+| 0 | 0 | 55332.9297 | 57511.9648 | 333834.9375 | 63.8516 | 46.984 | 11.746 | 57797.4375 |
+| 500 | 0.0269 | 2446.0188 | 10865.3799 | 11817.8984 | 64.1124 | 46.793 | 11.698 | 39870.7812 |
+| 1000 | 0.0539 | 1804.3785 | 6767.0361 | 9836.2031 | 64.095 | 46.806 | 11.701 | 19923.8262 |
+| 1500 | 0.0808 | 1456.1499 | 5625.2583 | 9170.7520 | 65.271 | 45.962 | 11.491 | 18979.7988 |
+| 2000 | 0.1077 | 1255.2349 | 5859.0298 | 8742.0908 | 64.4753 | 46.529 | 11.632 | 17829.9570 |
+| 2500 | 0.1347 | 1123.1558 | 5142.7266 | 8474.3467 | 64.6172 | 46.427 | 11.607 | 18204.0723 |
+| 3000 | 0.1616 | 1041.0769 | 5179.4790 | 8192.3838 | 64.0965 | 46.804 | 11.701 | 16922.875 |
+| 3500 | 0.1886 | 948.0062 | 4929.7056 | 7911.0400 | 64.5488 | 46.476 | 11.619 | 22088.8789 |
+| 4000 | 0.2155 | 899.1066 | 4752.2407 | 7641.6426 | 65.5993 | 45.732 | 11.433 | 16942.1074 |
+| 4500 | 0.2424 | 843.3125 | 4732.0117 | 7480.7788 | 64.5158 | 46.5 | 11.625 | 13217.6758 |
+| 5000 | 0.2694 | 796.5746 | 4456.2817 | 7343.6479 | 65.0161 | 46.142 | 11.536 | 12772.6074 |
+| 5500 | 0.2963 | 772.2271 | 4386.7627 | 7222.3145 | 65.0008 | 46.153 | 11.538 | 11082.3330 |
+| 6000 | 0.3232 | 723.9974 | 4267.7817 | 7016.9600 | 64.7743 | 46.315 | 11.579 | 9581.7812 |
+| 6500 | 0.3502 | 696.7773 | 4287.5391 | 6892.1387 | 64.6727 | 46.387 | 11.597 | 8422.7246 |
+| 7000 | 0.3771 | 679.4652 | 4046.8250 | 6773.9629 | 64.6977 | 46.369 | 11.592 | 7275.9604 |
+| 7500 | 0.4040 | 667.8522 | 4138.4370 | 6713.6533 | 65.028 | 46.134 | 11.533 | 8175.5986 |
+| 8000 | 0.4310 | 647.4772 | 3977.0999 | 6626.9331 | 64.3886 | 46.592 | 11.648 | 5914.0166 |
+| 8500 | 0.4579 | 627.8210 | 3850.3174 | 6548.4160 | 64.3532 | 46.618 | 11.654 | 7728.6548 |
+| 9000 | 0.4848 | 608.0646 | 3773.8511 | 6449.4614 | 64.3549 | 46.616 | 11.654 | 7419.7065 |
+| 9500 | 0.5118 | 603.2673 | 3866.3679 | 6355.0508 | 64.6366 | 46.413 | 11.603 | 9060.9883 |
+| 10000 | 0.5387 | 588.2559 | 3563.7371 | 6282.0479 | 65.1489 | 46.048 | 11.512 | 7187.1206 |
+| 10500 | 0.5657 | 569.4130 | 3654.1926 | 6309.9839 | 64.8852 | 46.235 | 11.559 | 7732.7837 |
+| 11000 | 0.5926 | 572.8280 | 3728.8887 | 6206.9868 | 65.1196 | 46.069 | 11.517 | 6973.9194 |
+| 11500 | 0.6195 | 551.1736 | 3640.4358 | 6146.9331 | 65.3439 | 45.911 | 11.478 | 5983.9292 |
+| 12000 | 0.6465 | 544.3150 | 3507.0312 | 6073.0454 | 65.3717 | 45.891 | 11.473 | 5726.3408 |
+| 12500 | 0.6734 | 538.8688 | 3312.2402 | 6032.6079 | 65.1968 | 46.015 | 11.504 | 5642.0854 |
+| 13000 | 0.7003 | 525.2048 | 3317.0325 | 6042.6240 | 65.216 | 46.001 | 11.5 | 11299.7695 |
+| 13500 | 0.7273 | 516.2283 | 3381.7358 | 5946.3682 | 67.4205 | 44.497 | 11.124 | 7501.9004 |
+| 14000 | 0.7542 | 508.5393 | 3201.6807 | 5921.8345 | 65.0932 | 46.088 | 11.522 | 7485.8843 |
+| 14500 | 0.7811 | 499.8382 | 3091.7612 | 5887.8721 | 65.2716 | 45.962 | 11.49 | 5927.4609 |
+| 15000 | 0.8081 | 491.9155 | 3132.6841 | 5930.3252 | 65.4781 | 45.817 | 11.454 | 7431.6040 |
+| 15500 | 0.8350 | 485.9736 | 3050.2964 | 5844.8960 | 65.1349 | 46.058 | 11.515 | 6106.6260 |
+| 16000 | 0.8620 | 483.0016 | 2964.3213 | 5828.2241 | 65.6241 | 45.715 | 11.429 | 5001.1572 |
+| 16500 | 0.8889 | 480.1220 | 2957.3284 | 5789.1626 | 65.5498 | 45.767 | 11.442 | 4932.5088 |
+| 17000 | 0.9158 | 470.7449 | 2851.3689 | 5783.3174 | 65.2632 | 45.968 | 11.492 | 4651.6655 |
+| 17500 | 0.9428 | 471.4951 | 2821.2729 | 5762.3945 | 65.89 | 45.53 | 11.383 | 4335.2710 |
+| 18000 | 0.9697 | 467.4575 | 2898.3936 | 5772.5654 | 65.0307 | 46.132 | 11.533 | 3703.9866 |
+| 18500 | 0.9966 | 465.3025 | 2792.5769 | 5640.9438 | 65.1725 | 46.032 | 11.508 | 4174.7715 |
+| 18562 | 1.0000 | 459.9143 | 2775.4995 | 5699.1255 | 65.9141 | 45.514 | 11.378 | 4052.5564 |
 ### Framework versions
 - Distily 0.2.0