bert-base-uncased_08112024T161410

This model is a fine-tuned version of google-bert/bert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4679
• F1: 0.8733
• Learning Rate: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 600
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Rate
No log	0.9942	86	1.5646	0.3444	0.0000
No log	2.0	173	1.1261	0.5492	0.0000
No log	2.9942	259	0.8378	0.6911	0.0000
No log	4.0	346	0.7016	0.7403	0.0001
No log	4.9942	432	0.5643	0.8175	0.0001
0.9953	6.0	519	0.4947	0.8366	0.0001
0.9953	6.9942	605	0.5260	0.8488	0.0001
0.9953	8.0	692	0.6001	0.8525	0.0001
0.9953	8.9942	778	0.4679	0.8733	0.0001
0.9953	10.0	865	0.5398	0.8779	0.0001
0.9953	10.9942	951	0.5633	0.8765	0.0001
0.2162	12.0	1038	0.6918	0.8640	0.0001
0.2162	12.9942	1124	0.7215	0.8725	0.0001
0.2162	14.0	1211	0.7590	0.8755	0.0001
0.2162	14.9942	1297	0.8360	0.8775	0.0001
0.2162	16.0	1384	0.8535	0.8766	0.0001
0.2162	16.9942	1470	0.8997	0.8743	0.0001
0.059	18.0	1557	0.8916	0.8770	0.0001
0.059	18.9942	1643	0.8122	0.8835	0.0000
0.059	20.0	1730	0.8315	0.8852	0.0000
0.059	20.9942	1816	0.8983	0.8783	0.0000
0.059	22.0	1903	0.8399	0.8886	0.0000
0.059	22.9942	1989	0.8251	0.8887	0.0000
0.0109	24.0	2076	0.8765	0.8845	0.0000
0.0109	24.9942	2162	0.8813	0.8844	0.0000
0.0109	26.0	2249	0.8872	0.8850	0.0000
0.0109	26.9942	2335	0.8876	0.8851	0.0000
0.0109	28.0	2422	0.8902	0.8853	0.0000
0.0025	28.9942	2508	0.8904	0.8853	4e-07
0.0025	29.8266	2580	0.8904	0.8853	0.0

Framework versions

• Transformers 4.44.2
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.19.1