bert-suicide-detection-hk-large-new

This model is a fine-tuned version of hon9kon9ize/bert-large-cantonese on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5925
• Accuracy: 0.8987

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.5908	0.0573	20	0.4476	0.7975
0.543	0.1146	40	0.4423	0.7722
0.5093	0.1719	60	0.5882	0.7722
0.5186	0.2292	80	0.6422	0.7658
0.502	0.2865	100	0.9382	0.7658
0.573	0.3438	120	0.4264	0.8228
0.5269	0.4011	140	0.5453	0.8481
0.3545	0.4585	160	0.4541	0.8924
0.4449	0.5158	180	0.4354	0.8924
0.3868	0.5731	200	0.8784	0.8481
0.7576	0.6304	220	0.3822	0.8861
0.1956	0.6877	240	0.4668	0.8797
0.4942	0.7450	260	0.5736	0.8481
0.4762	0.8023	280	0.2911	0.8987
0.4136	0.8596	300	0.3629	0.8608
0.5865	0.9169	320	0.9794	0.7722
0.3758	0.9742	340	0.4678	0.8734
0.4285	1.0315	360	0.5543	0.8671
0.44	1.0888	380	0.5150	0.8608
0.3573	1.1461	400	0.5635	0.8608
0.4187	1.2034	420	0.6609	0.8481
0.3742	1.2607	440	0.5913	0.8481
0.5179	1.3181	460	0.3984	0.8354
0.1685	1.3754	480	0.5607	0.8734
0.5284	1.4327	500	0.3528	0.8924
0.4246	1.4900	520	0.5857	0.8608
0.2419	1.5473	540	0.3496	0.9051
0.4416	1.6046	560	0.4946	0.8861
0.4426	1.6619	580	0.3458	0.9051
0.2122	1.7192	600	0.6184	0.8987
0.1734	1.7765	620	0.7278	0.8734
0.2314	1.8338	640	0.5430	0.8861
0.4886	1.8911	660	0.5081	0.8861
0.3429	1.9484	680	0.6000	0.8481
0.3591	2.0057	700	0.5184	0.8608
0.3638	2.0630	720	0.4008	0.8861
0.1881	2.1203	740	0.6161	0.8734
0.241	2.1777	760	0.5249	0.8861
0.4699	2.2350	780	0.5323	0.8861
0.3702	2.2923	800	0.7284	0.8481
0.4192	2.3496	820	0.3671	0.9051
0.1747	2.4069	840	0.4293	0.9051
0.347	2.4642	860	0.4047	0.8924
0.0533	2.5215	880	0.5135	0.8861
0.2002	2.5788	900	0.5535	0.8797
0.0274	2.6361	920	0.6635	0.8734
0.2339	2.6934	940	0.4940	0.8924
0.3015	2.7507	960	0.5514	0.8734
0.4222	2.8080	980	0.5412	0.8734
0.3243	2.8653	1000	0.5440	0.8734
0.3137	2.9226	1020	0.4534	0.8861
0.191	2.9799	1040	0.6083	0.8797
0.1213	3.0372	1060	0.5798	0.8734
0.1582	3.0946	1080	0.4830	0.8861
0.0546	3.1519	1100	0.7039	0.8734
0.0387	3.2092	1120	0.6059	0.8924
0.4619	3.2665	1140	0.6934	0.8861
0.2789	3.3238	1160	0.5247	0.9051
0.1361	3.3811	1180	0.6307	0.8797
0.0475	3.4384	1200	0.5455	0.8924
0.2889	3.4957	1220	0.5865	0.8797
0.2507	3.5530	1240	0.5029	0.8861
0.1476	3.6103	1260	0.6517	0.8797
0.0709	3.6676	1280	0.5607	0.8797
0.2416	3.7249	1300	0.6906	0.8671
0.2482	3.7822	1320	0.4523	0.8987
0.1591	3.8395	1340	0.3677	0.9177
0.1728	3.8968	1360	0.4237	0.9051
0.1061	3.9542	1380	0.3708	0.9241
0.1461	4.0115	1400	0.4642	0.9051
0.0671	4.0688	1420	0.5567	0.8924
0.0363	4.1261	1440	0.6240	0.8861
0.1257	4.1834	1460	0.7054	0.8734
0.1307	4.2407	1480	0.6526	0.8861
0.226	4.2980	1500	0.5883	0.8797
0.0714	4.3553	1520	0.5382	0.8987
0.0617	4.4126	1540	0.6030	0.8924
0.0802	4.4699	1560	0.5677	0.8924
0.2404	4.5272	1580	0.5837	0.8987
0.2311	4.5845	1600	0.6192	0.8987
0.0031	4.6418	1620	0.6153	0.8987
0.1621	4.6991	1640	0.6008	0.8924
0.0841	4.7564	1660	0.5887	0.8987
0.0014	4.8138	1680	0.5866	0.8987
0.1199	4.8711	1700	0.5909	0.8987
0.0124	4.9284	1720	0.5906	0.8987
0.046	4.9857	1740	0.5925	0.8987

Framework versions

• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0