galsenai/wav2vec2-base-waxal-keyword-spotting

wav2vec2-base

This model is a fine-tuned version of facebook/wav2vec2-base on the galsenai/waxal_dataset dataset. It achieves the following results on the evaluation set:

• Loss: 0.7504
• Accuracy: 0.8632
• Precision: 0.9380
• F1: 0.8954

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 30
• eval_batch_size: 30
• seed: 0
• gradient_accumulation_steps: 4
• total_train_batch_size: 120
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 32.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	F1
4.3647	2.53	500	4.8202	0.0117	0.0134	0.0032
2.6202	5.05	1000	4.2238	0.0625	0.0781	0.0355
1.38	7.58	1500	3.6392	0.2941	0.5211	0.3174
0.8601	10.1	2000	2.7953	0.4907	0.7446	0.5657
0.5645	12.63	2500	1.9829	0.6862	0.8363	0.7421
0.4009	15.15	3000	1.4535	0.7635	0.9000	0.8174
0.3054	17.68	3500	1.1426	0.7882	0.9058	0.8298
0.2448	20.2	4000	0.9860	0.8189	0.9206	0.8593
0.2116	22.73	4500	0.8820	0.8325	0.9261	0.8711
0.1863	25.25	5000	0.8191	0.8465	0.9366	0.8848
0.1701	27.78	5500	0.7504	0.8632	0.9380	0.8954
0.1558	30.3	6000	0.7665	0.8609	0.9398	0.8956

Framework versions

• Transformers 4.27.0.dev0
• Pytorch 1.11.0+cu113
• Datasets 2.9.1.dev0
• Tokenizers 0.13.2