--- library_name: transformers license: cc-by-4.0 base_model: hon9kon9ize/bert-large-cantonese tags: - generated_from_trainer metrics: - accuracy model-index: - name: bert-suicide-detection-hk-large-new results: [] --- # bert-suicide-detection-hk-large-new This model is a fine-tuned version of [hon9kon9ize/bert-large-cantonese](https://huggingface.co/hon9kon9ize/bert-large-cantonese) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.5925 - Accuracy: 0.8987 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 4 - eval_batch_size: 4 - seed: 42 - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: linear - num_epochs: 5 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | |:-------------:|:------:|:----:|:---------------:|:--------:| | 0.5908 | 0.0573 | 20 | 0.4476 | 0.7975 | | 0.543 | 0.1146 | 40 | 0.4423 | 0.7722 | | 0.5093 | 0.1719 | 60 | 0.5882 | 0.7722 | | 0.5186 | 0.2292 | 80 | 0.6422 | 0.7658 | | 0.502 | 0.2865 | 100 | 0.9382 | 0.7658 | | 0.573 | 0.3438 | 120 | 0.4264 | 0.8228 | | 0.5269 | 0.4011 | 140 | 0.5453 | 0.8481 | | 0.3545 | 0.4585 | 160 | 0.4541 | 0.8924 | | 0.4449 | 0.5158 | 180 | 0.4354 | 0.8924 | | 0.3868 | 0.5731 | 200 | 0.8784 | 0.8481 | | 0.7576 | 0.6304 | 220 | 0.3822 | 0.8861 | | 0.1956 | 0.6877 | 240 | 0.4668 | 0.8797 | | 0.4942 | 0.7450 | 260 | 0.5736 | 0.8481 | | 0.4762 | 0.8023 | 280 | 0.2911 | 0.8987 | | 0.4136 | 0.8596 | 300 | 0.3629 | 0.8608 | | 0.5865 | 0.9169 | 320 | 0.9794 | 0.7722 | | 0.3758 | 0.9742 | 340 | 0.4678 | 0.8734 | | 0.4285 | 1.0315 | 360 | 0.5543 | 0.8671 | | 0.44 | 1.0888 | 380 | 0.5150 | 0.8608 | | 0.3573 | 1.1461 | 400 | 0.5635 | 0.8608 | | 0.4187 | 1.2034 | 420 | 0.6609 | 0.8481 | | 0.3742 | 1.2607 | 440 | 0.5913 | 0.8481 | | 0.5179 | 1.3181 | 460 | 0.3984 | 0.8354 | | 0.1685 | 1.3754 | 480 | 0.5607 | 0.8734 | | 0.5284 | 1.4327 | 500 | 0.3528 | 0.8924 | | 0.4246 | 1.4900 | 520 | 0.5857 | 0.8608 | | 0.2419 | 1.5473 | 540 | 0.3496 | 0.9051 | | 0.4416 | 1.6046 | 560 | 0.4946 | 0.8861 | | 0.4426 | 1.6619 | 580 | 0.3458 | 0.9051 | | 0.2122 | 1.7192 | 600 | 0.6184 | 0.8987 | | 0.1734 | 1.7765 | 620 | 0.7278 | 0.8734 | | 0.2314 | 1.8338 | 640 | 0.5430 | 0.8861 | | 0.4886 | 1.8911 | 660 | 0.5081 | 0.8861 | | 0.3429 | 1.9484 | 680 | 0.6000 | 0.8481 | | 0.3591 | 2.0057 | 700 | 0.5184 | 0.8608 | | 0.3638 | 2.0630 | 720 | 0.4008 | 0.8861 | | 0.1881 | 2.1203 | 740 | 0.6161 | 0.8734 | | 0.241 | 2.1777 | 760 | 0.5249 | 0.8861 | | 0.4699 | 2.2350 | 780 | 0.5323 | 0.8861 | | 0.3702 | 2.2923 | 800 | 0.7284 | 0.8481 | | 0.4192 | 2.3496 | 820 | 0.3671 | 0.9051 | | 0.1747 | 2.4069 | 840 | 0.4293 | 0.9051 | | 0.347 | 2.4642 | 860 | 0.4047 | 0.8924 | | 0.0533 | 2.5215 | 880 | 0.5135 | 0.8861 | | 0.2002 | 2.5788 | 900 | 0.5535 | 0.8797 | | 0.0274 | 2.6361 | 920 | 0.6635 | 0.8734 | | 0.2339 | 2.6934 | 940 | 0.4940 | 0.8924 | | 0.3015 | 2.7507 | 960 | 0.5514 | 0.8734 | | 0.4222 | 2.8080 | 980 | 0.5412 | 0.8734 | | 0.3243 | 2.8653 | 1000 | 0.5440 | 0.8734 | | 0.3137 | 2.9226 | 1020 | 0.4534 | 0.8861 | | 0.191 | 2.9799 | 1040 | 0.6083 | 0.8797 | | 0.1213 | 3.0372 | 1060 | 0.5798 | 0.8734 | | 0.1582 | 3.0946 | 1080 | 0.4830 | 0.8861 | | 0.0546 | 3.1519 | 1100 | 0.7039 | 0.8734 | | 0.0387 | 3.2092 | 1120 | 0.6059 | 0.8924 | | 0.4619 | 3.2665 | 1140 | 0.6934 | 0.8861 | | 0.2789 | 3.3238 | 1160 | 0.5247 | 0.9051 | | 0.1361 | 3.3811 | 1180 | 0.6307 | 0.8797 | | 0.0475 | 3.4384 | 1200 | 0.5455 | 0.8924 | | 0.2889 | 3.4957 | 1220 | 0.5865 | 0.8797 | | 0.2507 | 3.5530 | 1240 | 0.5029 | 0.8861 | | 0.1476 | 3.6103 | 1260 | 0.6517 | 0.8797 | | 0.0709 | 3.6676 | 1280 | 0.5607 | 0.8797 | | 0.2416 | 3.7249 | 1300 | 0.6906 | 0.8671 | | 0.2482 | 3.7822 | 1320 | 0.4523 | 0.8987 | | 0.1591 | 3.8395 | 1340 | 0.3677 | 0.9177 | | 0.1728 | 3.8968 | 1360 | 0.4237 | 0.9051 | | 0.1061 | 3.9542 | 1380 | 0.3708 | 0.9241 | | 0.1461 | 4.0115 | 1400 | 0.4642 | 0.9051 | | 0.0671 | 4.0688 | 1420 | 0.5567 | 0.8924 | | 0.0363 | 4.1261 | 1440 | 0.6240 | 0.8861 | | 0.1257 | 4.1834 | 1460 | 0.7054 | 0.8734 | | 0.1307 | 4.2407 | 1480 | 0.6526 | 0.8861 | | 0.226 | 4.2980 | 1500 | 0.5883 | 0.8797 | | 0.0714 | 4.3553 | 1520 | 0.5382 | 0.8987 | | 0.0617 | 4.4126 | 1540 | 0.6030 | 0.8924 | | 0.0802 | 4.4699 | 1560 | 0.5677 | 0.8924 | | 0.2404 | 4.5272 | 1580 | 0.5837 | 0.8987 | | 0.2311 | 4.5845 | 1600 | 0.6192 | 0.8987 | | 0.0031 | 4.6418 | 1620 | 0.6153 | 0.8987 | | 0.1621 | 4.6991 | 1640 | 0.6008 | 0.8924 | | 0.0841 | 4.7564 | 1660 | 0.5887 | 0.8987 | | 0.0014 | 4.8138 | 1680 | 0.5866 | 0.8987 | | 0.1199 | 4.8711 | 1700 | 0.5909 | 0.8987 | | 0.0124 | 4.9284 | 1720 | 0.5906 | 0.8987 | | 0.046 | 4.9857 | 1740 | 0.5925 | 0.8987 | ### Framework versions - Transformers 4.48.3 - Pytorch 2.5.1+cu124 - Datasets 3.3.2 - Tokenizers 0.21.0