xezpeleta commited on
Commit
de3cb6e
·
verified ·
1 Parent(s): 850de1e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +54 -39
README.md CHANGED
@@ -16,70 +16,85 @@ model-index:
16
  name: Automatic Speech Recognition
17
  type: automatic-speech-recognition
18
  dataset:
19
- name: asierhv/composite_corpus_eu_v2.1
20
- type: asierhv/composite_corpus_eu_v2.1
21
  metrics:
22
  - name: Wer
23
  type: wer
24
- value: 12.307980517047582
 
 
25
  ---
26
 
27
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
28
- should probably proofread and complete it, then remove this comment. -->
29
-
30
  # Whisper Base Basque
31
 
32
- This model is a fine-tuned version of [openai/whisper-base](https://huggingface.co/openai/whisper-base) on the asierhv/composite_corpus_eu_v2.1 dataset.
33
- It achieves the following results on the evaluation set:
34
- - Loss: 0.3183
35
- - Wer: 12.3080
 
 
36
 
37
  ## Model description
38
 
39
- More information needed
40
 
41
  ## Intended uses & limitations
42
 
43
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
44
 
45
  ## Training and evaluation data
46
 
47
- More information needed
 
48
 
49
  ## Training procedure
50
 
51
  ### Training hyperparameters
52
 
53
  The following hyperparameters were used during training:
54
- - learning_rate: 2.5e-05
55
- - train_batch_size: 32
56
- - eval_batch_size: 16
57
- - seed: 42
58
- - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
59
- - lr_scheduler_type: linear
60
- - lr_scheduler_warmup_steps: 500
61
- - training_steps: 10000
62
- - mixed_precision_training: Native AMP
63
 
64
- ### Training results
 
 
 
 
 
 
 
 
65
 
66
- | Training Loss | Epoch | Step | Validation Loss | Wer |
67
- |:-------------:|:-----:|:-----:|:---------------:|:-------:|
68
- | 0.4816 | 0.1 | 1000 | 0.5136 | 25.7525 |
69
- | 0.2515 | 0.2 | 2000 | 0.4336 | 19.9950 |
70
- | 0.1792 | 0.3 | 3000 | 0.4054 | 17.6408 |
71
- | 0.2485 | 0.4 | 4000 | 0.3804 | 16.3794 |
72
- | 0.1007 | 0.5 | 5000 | 0.4056 | 15.2554 |
73
- | 0.1296 | 0.6 | 6000 | 0.3731 | 15.3241 |
74
- | 0.1555 | 0.7 | 7000 | 0.3764 | 13.3820 |
75
- | 0.114 | 0.8 | 8000 | 0.3097 | 12.7513 |
76
- | 0.0775 | 0.9 | 9000 | 0.3170 | 12.4578 |
77
- | 0.0836 | 1.0 | 10000 | 0.3183 | 12.3080 |
78
 
 
 
 
 
 
 
 
 
 
 
 
 
79
 
80
  ### Framework versions
81
 
82
- - Transformers 4.49.0.dev0
83
- - Pytorch 2.6.0+cu124
84
- - Datasets 3.3.1.dev0
85
- - Tokenizers 0.21.0
 
16
  name: Automatic Speech Recognition
17
  type: automatic-speech-recognition
18
  dataset:
19
+ name: Mozilla Common Voice 18.0
20
+ type: mozilla-foundation/common_voice_18_0
21
  metrics:
22
  - name: Wer
23
  type: wer
24
+ value: 10.78
25
+ language:
26
+ - eu
27
  ---
28
 
 
 
 
29
  # Whisper Base Basque
30
 
31
+ This model is a fine-tuned version of [openai/whisper-base](https://huggingface.co/openai/whisper-base) specifically for Basque (eu) language Automatic Speech Recognition (ASR). It was trained on the [asierhv/composite_corpus_eu_v2.1](https://huggingface.co/datasets/asierhv/composite_corpus_eu_v2.1) dataset, which is a composite corpus designed to improve Basque ASR performance.
32
+
33
+ **Key improvements and results compared to the base model:**
34
+
35
+ * **Significant WER reduction:** The fine-tuned model achieves a Word Error Rate (WER) of 12.3080 on the validation set of the `asierhv/composite_corpus_eu_v2.1` dataset, demonstrating improved accuracy compared to the base `whisper-base` model for Basque.
36
+ * **Performance on Common Voice:** When evaluated on the Mozilla Common Voice 18.0 dataset, the model achieved a WER of 10.78. This demonstrates the model's ability to generalize to other Basque speech datasets, and highlights the improvement in accuracy due to the larger base model.
37
 
38
  ## Model description
39
 
40
+ This model builds upon the `whisper-base` architecture, known for its strong performance in multilingual speech recognition. By fine-tuning this model on a dedicated Basque speech corpus, it specializes in accurately transcribing Basque speech. The `whisper-base` model offers a larger capacity than `whisper-tiny`, resulting in higher accuracy, albeit with increased computational requirements.
41
 
42
  ## Intended uses & limitations
43
 
44
+ **Intended uses:**
45
+
46
+ * High-accuracy automatic transcription of Basque speech.
47
+ * Development of advanced Basque speech-based applications.
48
+ * Research in Basque speech processing requiring higher accuracy.
49
+ * Professional transcription services for Basque language.
50
+ * Applications where slightly higher computational cost is acceptable for improved accuracy.
51
+
52
+ **Limitations:**
53
+
54
+ * Performance remains dependent on audio quality, with challenges posed by background noise and poor recording conditions.
55
+ * Accuracy may still be affected by highly dialectal or informal Basque speech.
56
+ * While demonstrating improved performance, the model may still produce errors, especially with complex linguistic structures.
57
+ * The base model is larger than the tiny, so inference will be slower and require more resources.
58
 
59
  ## Training and evaluation data
60
 
61
+ * **Training dataset:** [asierhv/composite_corpus_eu_v2.1](https://huggingface.co/datasets/asierhv/composite_corpus_eu_v2.1). This dataset is a carefully curated compilation of Basque speech data, designed to enhance the effectiveness of Basque ASR systems.
62
+ * **Evaluation Dataset:** The `test` portion of `asierhv/composite_corpus_eu_v2.1`.
63
 
64
  ## Training procedure
65
 
66
  ### Training hyperparameters
67
 
68
  The following hyperparameters were used during training:
 
 
 
 
 
 
 
 
 
69
 
70
+ * **learning_rate:** 2.5e-05
71
+ * **train_batch_size:** 32
72
+ * **eval_batch_size:** 16
73
+ * **seed:** 42
74
+ * **optimizer:** AdamW with betas=(0.9, 0.999) and epsilon=1e-08
75
+ * **lr_scheduler_type:** linear
76
+ * **lr_scheduler_warmup_steps:** 500
77
+ * **training_steps:** 10000
78
+ * **mixed_precision_training:** Native AMP
79
 
80
+ ### Training results
 
 
 
 
 
 
 
 
 
 
 
81
 
82
+ | Training Loss | Epoch | Step | Validation Loss | WER |
83
+ |---------------|-------|-------|-----------------|----------|
84
+ | 0.4816 | 0.1 | 1000 | 0.5136 | 25.7525 |
85
+ | 0.2515 | 0.2 | 2000 | 0.4336 | 19.9950 |
86
+ | 0.1792 | 0.3 | 3000 | 0.4054 | 17.6408 |
87
+ | 0.2485 | 0.4 | 4000 | 0.3804 | 16.3794 |
88
+ | 0.1007 | 0.5 | 5000 | 0.4056 | 15.2554 |
89
+ | 0.1296 | 0.6 | 6000 | 0.3731 | 15.3241 |
90
+ | 0.1555 | 0.7 | 7000 | 0.3764 | 13.3820 |
91
+ | 0.114 | 0.8 | 8000 | 0.3097 | 12.7513 |
92
+ | 0.0775 | 0.9 | 9000 | 0.3170 | 12.4578 |
93
+ | 0.0836 | 1.0 | 10000 | 0.3183 | 12.3080 |
94
 
95
  ### Framework versions
96
 
97
+ * Transformers 4.49.0.dev0
98
+ * Pytorch 2.6.0+cu124
99
+ * Datasets 3.3.1.dev0
100
+ * Tokenizers 0.21.0