prft1

This model is a fine-tuned version of distilbert/distilgpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0107

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	1	0.0498
No log	2.0	2	0.0476
No log	3.0	3	0.0462
No log	4.0	4	0.0447
No log	5.0	5	0.0433
No log	6.0	6	0.0417
No log	7.0	7	0.0402
No log	8.0	8	0.0386
No log	9.0	9	0.0372
No log	10.0	10	0.0358
No log	11.0	11	0.0345
No log	12.0	12	0.0332
No log	13.0	13	0.0319
No log	14.0	14	0.0306
No log	15.0	15	0.0293
No log	16.0	16	0.0281
No log	17.0	17	0.0269
No log	18.0	18	0.0258
No log	19.0	19	0.0247
No log	20.0	20	0.0238
No log	21.0	21	0.0228
No log	22.0	22	0.0219
No log	23.0	23	0.0210
No log	24.0	24	0.0202
No log	25.0	25	0.0194
No log	26.0	26	0.0187
No log	27.0	27	0.0179
No log	28.0	28	0.0172
No log	29.0	29	0.0166
No log	30.0	30	0.0160
No log	31.0	31	0.0154
No log	32.0	32	0.0149
No log	33.0	33	0.0144
No log	34.0	34	0.0140
No log	35.0	35	0.0136
No log	36.0	36	0.0132
No log	37.0	37	0.0128
No log	38.0	38	0.0125
No log	39.0	39	0.0122
No log	40.0	40	0.0120
No log	41.0	41	0.0117
No log	42.0	42	0.0115
No log	43.0	43	0.0113
No log	44.0	44	0.0111
No log	45.0	45	0.0110
No log	46.0	46	0.0109
No log	47.0	47	0.0108
No log	48.0	48	0.0107
No log	49.0	49	0.0107
No log	50.0	50	0.0107

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2