mt5-small-finetuned-tq-to-ar

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.2607
• Bleu: 0.0533
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
11.4687	1.0	656	4.1640	0.0355	18.2007
5.255	2.0	1312	3.6826	0.0314	18.8971
4.6562	3.0	1968	3.5592	0.03	19.0
4.329	4.0	2624	3.4714	0.0304	19.0
4.2431	5.0	3280	3.4219	0.0425	19.0
4.1638	6.0	3936	3.3766	0.0532	19.0
4.0727	7.0	4592	3.3530	0.0539	19.0
4.0424	8.0	5248	3.3293	0.0536	19.0
3.9878	9.0	5904	3.3079	0.0526	19.0
3.9417	10.0	6560	3.2885	0.0532	19.0
3.9272	11.0	7216	3.2813	0.0531	19.0
3.9144	12.0	7872	3.2733	0.0522	19.0
3.8918	13.0	8528	3.2665	0.0539	19.0
3.8884	14.0	9184	3.2617	0.0527	19.0
3.8857	15.0	9840	3.2607	0.0533	19.0

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.4
• Tokenizers 0.13.3