finetuned-CK

This model is a fine-tuned version of microsoft/beit-base-patch16-224-pt22k-ft22k on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1159
• Accuracy: 0.9873

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.87	5	2.0523	0.1139
1.9841	1.91	11	1.9751	0.0633
1.9841	2.96	17	1.8640	0.1392
1.8698	4.0	23	1.7348	0.3291
1.8698	4.87	28	1.6262	0.4430
1.6947	5.91	34	1.4895	0.5823
1.6038	6.96	40	1.3500	0.5949
1.6038	8.0	46	1.1857	0.6835
1.4019	8.87	51	1.0636	0.6962
1.4019	9.91	57	0.9472	0.7468
1.2101	10.96	63	0.8263	0.7722
1.2101	12.0	69	0.7643	0.7595
1.077	12.87	74	0.7260	0.7722
0.9735	13.91	80	0.6628	0.8228
0.9735	14.96	86	0.6245	0.8101
0.8696	16.0	92	0.5579	0.8101
0.8696	16.87	97	0.5309	0.8228
0.858	17.91	103	0.5099	0.8354
0.858	18.96	109	0.4997	0.8354
0.7716	20.0	115	0.4679	0.8608
0.699	20.87	120	0.4455	0.8481
0.699	21.91	126	0.4409	0.8608
0.6768	22.96	132	0.4186	0.8481
0.6768	24.0	138	0.3826	0.8734
0.6303	24.87	143	0.3708	0.8861
0.6303	25.91	149	0.3545	0.8608
0.5786	26.96	155	0.3480	0.8987
0.5188	28.0	161	0.3241	0.9241
0.5188	28.87	166	0.3257	0.9114
0.5349	29.91	172	0.2963	0.9241
0.5349	30.96	178	0.2836	0.9367
0.5208	32.0	184	0.2822	0.9620
0.5208	32.87	189	0.2933	0.9494
0.4458	33.91	195	0.2742	0.9620
0.4716	34.96	201	0.2580	0.9620
0.4716	36.0	207	0.2432	0.9620
0.427	36.87	212	0.2333	0.9747
0.427	37.91	218	0.2115	0.9494
0.4052	38.96	224	0.2044	0.9873
0.3899	40.0	230	0.2081	0.9747
0.3899	40.87	235	0.2110	0.9620
0.4098	41.91	241	0.2099	0.9494
0.4098	42.96	247	0.1899	0.9747
0.3691	44.0	253	0.1783	0.9873
0.3691	44.87	258	0.1768	0.9747
0.3878	45.91	264	0.1836	0.9873
0.3582	46.96	270	0.1876	0.9747
0.3582	48.0	276	0.1761	0.9747
0.356	48.87	281	0.1745	0.9620
0.356	49.91	287	0.1761	0.9620
0.3715	50.96	293	0.1716	0.9873
0.3715	52.0	299	0.1694	0.9620
0.3195	52.87	304	0.1660	0.9873
0.3452	53.91	310	0.1584	0.9873
0.3452	54.96	316	0.1579	0.9620
0.3407	56.0	322	0.1431	0.9873
0.3407	56.87	327	0.1423	0.9873
0.3092	57.91	333	0.1434	0.9747
0.3092	58.96	339	0.1391	0.9873
0.3346	60.0	345	0.1362	0.9873
0.3107	60.87	350	0.1341	0.9873
0.3107	61.91	356	0.1368	0.9873
0.2884	62.96	362	0.1393	0.9747
0.2884	64.0	368	0.1347	0.9873
0.3048	64.87	373	0.1334	0.9873
0.3048	65.91	379	0.1342	0.9747
0.3452	66.96	385	0.1310	0.9747
0.2835	68.0	391	0.1308	0.9873
0.2835	68.87	396	0.1304	0.9747
0.3865	69.91	402	0.1264	0.9873
0.3865	70.96	408	0.1247	0.9873
0.2729	72.0	414	0.1239	0.9873
0.2729	72.87	419	0.1249	0.9747
0.2643	73.91	425	0.1189	0.9873
0.3176	74.96	431	0.1171	0.9873
0.3176	76.0	437	0.1174	0.9873
0.3184	76.87	442	0.1184	0.9873
0.3184	77.91	448	0.1160	0.9873
0.2817	78.96	454	0.1147	0.9873
0.2666	80.0	460	0.1133	0.9873
0.2666	80.87	465	0.1139	0.9873
0.2589	81.91	471	0.1142	0.9873
0.2589	82.96	477	0.1150	0.9873
0.275	84.0	483	0.1156	0.9873
0.275	84.87	488	0.1160	0.9873
0.2644	85.91	494	0.1160	0.9873
0.3187	86.96	500	0.1159	0.9873

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0