scenario-kd-pre-ner-full-xlmr_data-univner_half66

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 54.2999
• Precision: 0.7915
• Recall: 0.7915
• F1: 0.7915
• Accuracy: 0.9788

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 32
• seed: 66
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
117.0453	0.5828	500	83.3941	0.7646	0.6520	0.7038	0.9712
73.6271	1.1655	1000	71.5096	0.7579	0.7647	0.7613	0.9767
64.6895	1.7483	1500	67.1628	0.7700	0.7426	0.7561	0.9760
60.195	2.3310	2000	64.5600	0.7847	0.7155	0.7485	0.9754
57.2113	2.9138	2500	61.9201	0.7835	0.7762	0.7798	0.9782
54.8514	3.4965	3000	60.4216	0.7799	0.7807	0.7803	0.9778
53.1617	4.0793	3500	59.1528	0.7942	0.7803	0.7872	0.9788
51.5153	4.6620	4000	58.0760	0.7879	0.7846	0.7862	0.9785
50.4122	5.2448	4500	57.1617	0.7821	0.7886	0.7853	0.9785
49.329	5.8275	5000	56.6207	0.7875	0.7785	0.7830	0.9783
48.5363	6.4103	5500	56.0604	0.7939	0.7771	0.7854	0.9786
47.9093	6.9930	6000	55.3932	0.7935	0.7743	0.7838	0.9784
47.3113	7.5758	6500	55.0312	0.7998	0.7745	0.7869	0.9787
46.8636	8.1585	7000	54.6807	0.7915	0.7885	0.7900	0.9790
46.5397	8.7413	7500	54.4381	0.7909	0.7882	0.7896	0.9786
46.3116	9.3240	8000	54.3428	0.7913	0.7888	0.7900	0.9789
46.2021	9.9068	8500	54.2999	0.7915	0.7915	0.7915	0.9788

Framework versions

• Transformers 4.44.2
• Pytorch 2.1.1+cu121
• Datasets 2.14.5
• Tokenizers 0.19.1