fine-tuned-DatasetQAS-TYDI-QA-ID-with-indobert-large-p2-with-ITTL-without-freeze-LR-1e-05

This model is a fine-tuned version of indobenchmark/indobert-large-p2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1732
• Exact Match: 60.2113
• F1: 73.6360

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1
6.3258	0.5	19	3.5172	10.7394	21.8207
6.3258	0.99	38	2.7869	18.3099	28.8508
3.5421	1.5	57	2.3403	23.4155	35.8166
3.5421	1.99	76	2.0205	30.2817	42.8646
3.5421	2.5	95	1.7202	38.7324	51.9337
2.0678	2.99	114	1.4839	44.5423	59.5131
2.0678	3.5	133	1.3583	50.5282	64.3127
1.4302	3.99	152	1.2706	52.2887	66.7363
1.4302	4.5	171	1.2389	55.1056	69.3028
1.4302	4.99	190	1.2065	55.8099	69.9972
1.0965	5.5	209	1.1840	56.8662	70.7353
1.0965	5.99	228	1.1887	58.4507	71.9049
1.0965	6.5	247	1.1797	58.6268	72.5495
0.9291	6.99	266	1.1685	59.6831	73.0495
0.9291	7.5	285	1.1713	59.1549	73.1298
0.839	7.99	304	1.1721	60.0352	73.0300
0.839	8.5	323	1.1735	60.0352	73.5065
0.839	8.99	342	1.1725	60.2113	73.6033
0.7854	9.5	361	1.1738	60.2113	73.6432
0.7854	9.99	380	1.1732	60.2113	73.6360

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2