train_3

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1714

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 36
• eval_batch_size: 36
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 10000
• num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss
0.1475	1.0	1840	0.2238
0.1271	2.0	3680	0.2051
0.121	3.0	5520	0.1829
0.1119	4.0	7360	0.1797
0.1039	5.0	9200	0.1723
0.1017	6.0	11040	0.1705
0.0981	7.0	12880	0.1656
0.0966	8.0	14720	0.1684
0.0977	9.0	16560	0.1672
0.0946	10.0	18400	0.1701
0.0911	11.0	20240	0.1616
0.0956	12.0	22080	0.1666
0.0886	13.0	23920	0.1665
0.0909	14.0	25760	0.1653
0.0903	15.0	27600	0.1679
0.0865	16.0	29440	0.1645
0.0887	17.0	31280	0.1665
0.0863	18.0	33120	0.1661
0.0848	19.0	34960	0.1644
0.0845	20.0	36800	0.1673
0.0846	21.0	38640	0.1687
0.0823	22.0	40480	0.1669
0.0853	23.0	42320	0.1679
0.0827	24.0	44160	0.1682
0.0828	25.0	46000	0.1691
0.0798	26.0	47840	0.1714

Framework versions

• Transformers 4.44.0.dev0
• Pytorch 2.6.0.dev20241217
• Datasets 2.20.0
• Tokenizers 0.19.1