md_mt5_0109_v3

This model is a fine-tuned version of Buseak/md_mt5_0109_v2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1474
• Bleu: 0.582
• Gen Len: 18.9438

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
0.461	1.0	975	0.2323	0.5382	18.9456
0.434	2.0	1950	0.2157	0.5482	18.9446
0.4083	3.0	2925	0.2039	0.5526	18.949
0.3894	4.0	3900	0.1898	0.5577	18.9485
0.3766	5.0	4875	0.1827	0.5625	18.9508
0.3605	6.0	5850	0.1751	0.5665	18.9508
0.3497	7.0	6825	0.1680	0.5717	18.949
0.3325	8.0	7800	0.1634	0.5735	18.9423
0.323	9.0	8775	0.1581	0.574	18.9469
0.3211	10.0	9750	0.1546	0.58	18.9467
0.3177	11.0	10725	0.1526	0.5805	18.9464
0.3085	12.0	11700	0.1498	0.5831	18.9459
0.3056	13.0	12675	0.1485	0.5816	18.9456
0.304	14.0	13650	0.1478	0.5819	18.9438
0.3015	15.0	14625	0.1474	0.582	18.9438

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0