whisper-tiny-khmer-v8-aug-kcc

This model is a fine-tuned version of openai/whisper-tiny on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.9082
• Wer: 63.7181

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• lr_scheduler_warmup_steps: 1000
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.0673	1.0	901	0.8237	117.7249
0.9849	2.0	1802	0.6511	86.0226
0.7844	3.0	2703	0.5869	75.1971
0.6714	4.0	3604	0.5537	75.1320
0.5864	5.0	4505	0.5403	67.7720
0.5204	6.0	5406	0.5472	63.5759
0.4654	7.0	6307	0.5568	60.1273
0.4153	8.0	7208	0.5640	54.4071
0.3725	9.0	8109	0.5792	60.7013
0.3372	10.0	9010	0.5975	62.2423
0.3084	11.0	9911	0.6095	57.3323
0.2794	12.0	10812	0.6414	54.7423
0.2531	13.0	11713	0.6682	66.2045
0.2355	14.0	12614	0.6797	57.9449
0.2182	15.0	13515	0.7048	61.1643
0.2019	16.0	14416	0.7220	62.0662
0.1887	17.0	15317	0.7344	57.9786
0.1798	18.0	16218	0.7518	65.2278
0.1705	19.0	17119	0.7689	60.7109
0.1624	20.0	18020	0.7936	64.2318
0.1551	21.0	18921	0.8040	69.1996
0.1475	22.0	19822	0.8189	70.1666
0.1411	23.0	20723	0.8348	63.9834
0.135	24.0	21624	0.8421	72.1755
0.13	25.0	22525	0.8512	65.4086
0.126	26.0	23426	0.8738	64.3066
0.1207	27.0	24327	0.8753	62.9585
0.1166	28.0	25228	0.8964	61.5284
0.1129	29.0	26129	0.9016	62.6643
0.1102	30.0	27030	0.9082	63.7181

Framework versions

• Transformers 4.47.0
• Pytorch 2.3.1
• Datasets 3.2.0
• Tokenizers 0.21.0