text-translit-detector-ru

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0102
• Mean Distance: 0
• Max Distance: 1

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Mean Distance	Max Distance
0.2779	1.0	1050	0.3371	0	1
0.0346	2.0	2100	0.0173	0	1
0.0072	3.0	3150	0.0077	0	1
0.0057	4.0	4200	0.0093	0	1
0.005	5.0	5250	0.0053	0	1
0.002	6.0	6300	0.0056	0	1
0.002	7.0	7350	0.0091	0	1
0.0018	8.0	8400	0.0010	0	0
0.0044	9.0	9450	0.0043	0	1
0.0024	10.0	10500	0.0048	0	1
0.0032	11.0	11550	0.0023	0	1
0.0028	12.0	12600	0.0003	0	0
0.0005	13.0	13650	0.0008	0	0
0.0026	14.0	14700	0.0012	0	1
0.0011	15.0	15750	0.0171	0	1
0.0	16.0	16800	0.0120	0	1
0.0001	17.0	17850	0.0031	0	1
0.0007	18.0	18900	0.0165	0	1
0.001	19.0	19950	0.0194	0	1
0.0001	20.0	21000	0.0102	0	1

Framework versions

• Transformers 4.35.0
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1