furina_seed42_eng_esp_kin

This model is a fine-tuned version of yihongLiu/furina on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0131
• Spearman Corr: 0.8588

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 128
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Spearman Corr
No log	0.6	200	0.0342	0.6070
No log	1.21	400	0.0245	0.6934
No log	1.81	600	0.0230	0.7199
0.0458	2.42	800	0.0215	0.7448
0.0458	3.02	1000	0.0203	0.7510
0.0458	3.63	1200	0.0198	0.7712
0.02	4.23	1400	0.0180	0.7809
0.02	4.83	1600	0.0191	0.7812
0.02	5.44	1800	0.0182	0.7921
0.0142	6.04	2000	0.0177	0.8010
0.0142	6.65	2200	0.0170	0.8004
0.0142	7.25	2400	0.0159	0.8085
0.0142	7.85	2600	0.0161	0.8114
0.01	8.46	2800	0.0160	0.8142
0.01	9.06	3000	0.0152	0.8218
0.01	9.67	3200	0.0157	0.8234
0.0072	10.27	3400	0.0145	0.8303
0.0072	10.88	3600	0.0153	0.8311
0.0072	11.48	3800	0.0147	0.8311
0.0059	12.08	4000	0.0140	0.8373
0.0059	12.69	4200	0.0139	0.8401
0.0059	13.29	4400	0.0143	0.8406
0.0059	13.9	4600	0.0136	0.8447
0.0049	14.5	4800	0.0140	0.8453
0.0049	15.11	5000	0.0133	0.8452
0.0049	15.71	5200	0.0140	0.8450
0.0041	16.31	5400	0.0135	0.8481
0.0041	16.92	5600	0.0147	0.8489
0.0041	17.52	5800	0.0135	0.8492
0.0037	18.13	6000	0.0134	0.8498
0.0037	18.73	6200	0.0131	0.8492
0.0037	19.34	6400	0.0134	0.8524
0.0037	19.94	6600	0.0134	0.8536
0.0034	20.54	6800	0.0128	0.8540
0.0034	21.15	7000	0.0134	0.8539
0.0034	21.75	7200	0.0138	0.8531
0.0031	22.36	7400	0.0125	0.8562
0.0031	22.96	7600	0.0135	0.8585
0.0031	23.56	7800	0.0132	0.8569
0.0028	24.17	8000	0.0126	0.8564
0.0028	24.77	8200	0.0130	0.8574
0.0028	25.38	8400	0.0128	0.8587
0.0026	25.98	8600	0.0128	0.8595
0.0026	26.59	8800	0.0131	0.8582
0.0026	27.19	9000	0.0131	0.8588

Framework versions

• Transformers 4.37.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1