review_classification_bert_base_jp_v3_ratio1_1_2label_add_dropout-epoch40_v4

This model is a fine-tuned version of cl-tohoku/bert-base-japanese-v3 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.4594
• Accuracy: 0.8023

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	52	0.5181	0.7316
No log	2.0	104	0.4858	0.7684
No log	3.0	156	0.4934	0.7684
No log	4.0	208	0.5022	0.7910
No log	5.0	260	0.5365	0.7910
No log	6.0	312	0.6070	0.8136
No log	7.0	364	0.7252	0.8107
No log	8.0	416	0.9174	0.7825
No log	9.0	468	1.0906	0.7655
0.208	10.0	520	1.2154	0.7740
0.208	11.0	572	1.0851	0.8051
0.208	12.0	624	1.3699	0.7627
0.208	13.0	676	1.1715	0.7994
0.208	14.0	728	1.2354	0.7910
0.208	15.0	780	1.2615	0.7881
0.208	16.0	832	1.2891	0.7825
0.208	17.0	884	1.4250	0.7768
0.208	18.0	936	1.5752	0.7712
0.208	19.0	988	1.3456	0.8023
0.001	20.0	1040	1.3598	0.7994
0.001	21.0	1092	1.3378	0.8107
0.001	22.0	1144	1.3534	0.8051
0.001	23.0	1196	1.3832	0.7881
0.001	24.0	1248	1.3838	0.7938
0.001	25.0	1300	1.4257	0.7881
0.001	26.0	1352	1.4115	0.8023
0.001	27.0	1404	1.6230	0.7684
0.001	28.0	1456	1.4132	0.8023
0.0009	29.0	1508	1.6508	0.7684
0.0009	30.0	1560	1.4539	0.8023
0.0009	31.0	1612	1.4677	0.8023
0.0009	32.0	1664	1.4155	0.8079
0.0009	33.0	1716	1.4262	0.8079
0.0009	34.0	1768	1.4390	0.8023
0.0009	35.0	1820	1.4405	0.8023
0.0009	36.0	1872	1.4471	0.8051
0.0009	37.0	1924	1.4526	0.8023
0.0009	38.0	1976	1.4568	0.8023
0.0001	39.0	2028	1.4596	0.8023
0.0001	40.0	2080	1.4594	0.8023

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0