---
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- common_voice_11_0
metrics:
- wer
model-index:
- name: wav2vec2-large-xls-r-1b-swahili-v12
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: common_voice_11_0
      type: common_voice_11_0
      config: sw
      split: test
      args: sw
    metrics:
    - name: Wer
      type: wer
      value: 0.20382121671954753
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-large-xls-r-1b-swahili-v12

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-1b](https://huggingface.co/facebook/wav2vec2-xls-r-1b) on the common_voice_11_0 dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4658
- Wer: 0.2038

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 2.726         | 0.35  | 400   | 0.7214          | 0.6674 |
| 0.5241        | 0.69  | 800   | 0.5641          | 0.5345 |
| 0.4616        | 1.04  | 1200  | 0.5112          | 0.4755 |
| 0.4018        | 1.39  | 1600  | 0.4797          | 0.4158 |
| 0.3916        | 1.74  | 2000  | 0.4483          | 0.3985 |
| 0.3661        | 2.08  | 2400  | 0.4449          | 0.3931 |
| 0.3314        | 2.43  | 2800  | 0.4124          | 0.3549 |
| 0.3287        | 2.78  | 3200  | 0.4008          | 0.3651 |
| 0.317         | 3.13  | 3600  | 0.4460          | 0.3735 |
| 0.3026        | 3.47  | 4000  | 0.4165          | 0.3753 |
| 0.3061        | 3.82  | 4400  | 0.4112          | 0.3550 |
| 0.2808        | 4.17  | 4800  | 0.3951          | 0.3275 |
| 0.2641        | 4.52  | 5200  | 0.3934          | 0.3340 |
| 0.2709        | 4.86  | 5600  | 0.3963          | 0.3287 |
| 0.2586        | 5.21  | 6000  | 0.4114          | 0.3396 |
| 0.2487        | 5.56  | 6400  | 0.3821          | 0.3214 |
| 0.2618        | 5.91  | 6800  | 0.3987          | 0.3268 |
| 0.2297        | 6.25  | 7200  | 0.3810          | 0.3132 |
| 0.2337        | 6.6   | 7600  | 0.3740          | 0.3131 |
| 0.2285        | 6.95  | 8000  | 0.3715          | 0.3093 |
| 0.2173        | 7.29  | 8400  | 0.3878          | 0.3147 |
| 0.2251        | 7.64  | 8800  | 0.3862          | 0.3134 |
| 0.2215        | 7.99  | 9200  | 0.3621          | 0.2940 |
| 0.195         | 8.34  | 9600  | 0.3651          | 0.3005 |
| 0.201         | 8.68  | 10000 | 0.3837          | 0.3167 |
| 0.1964        | 9.03  | 10400 | 0.3719          | 0.2876 |
| 0.1741        | 9.38  | 10800 | 0.3637          | 0.2840 |
| 0.181         | 9.73  | 11200 | 0.3616          | 0.2914 |
| 0.1795        | 10.07 | 11600 | 0.3719          | 0.2753 |
| 0.1602        | 10.42 | 12000 | 0.3618          | 0.2856 |
| 0.1753        | 10.77 | 12400 | 0.3570          | 0.2788 |
| 0.1627        | 11.12 | 12800 | 0.3500          | 0.2719 |
| 0.1566        | 11.46 | 13200 | 0.3553          | 0.2808 |
| 0.1589        | 11.81 | 13600 | 0.3635          | 0.2699 |
| 0.1511        | 12.16 | 14000 | 0.3656          | 0.2692 |
| 0.1451        | 12.51 | 14400 | 0.3759          | 0.2759 |
| 0.1444        | 12.85 | 14800 | 0.3607          | 0.2677 |
| 0.1359        | 13.2  | 15200 | 0.3852          | 0.2660 |
| 0.1313        | 13.55 | 15600 | 0.3587          | 0.2679 |
| 0.1329        | 13.89 | 16000 | 0.3548          | 0.2584 |
| 0.1163        | 14.24 | 16400 | 0.3701          | 0.2535 |
| 0.1175        | 14.59 | 16800 | 0.3693          | 0.2638 |
| 0.1242        | 14.94 | 17200 | 0.3660          | 0.2565 |
| 0.1067        | 15.28 | 17600 | 0.3835          | 0.2581 |
| 0.1077        | 15.63 | 18000 | 0.3799          | 0.2504 |
| 0.1099        | 15.98 | 18400 | 0.3598          | 0.2478 |
| 0.0952        | 16.33 | 18800 | 0.3865          | 0.2563 |
| 0.1007        | 16.67 | 19200 | 0.3630          | 0.2565 |
| 0.0999        | 17.02 | 19600 | 0.3912          | 0.2505 |
| 0.0895        | 17.37 | 20000 | 0.3934          | 0.2631 |
| 0.0974        | 17.72 | 20400 | 0.3718          | 0.2462 |
| 0.0939        | 18.06 | 20800 | 0.4001          | 0.2587 |
| 0.0915        | 18.41 | 21200 | 0.4048          | 0.2468 |
| 0.0865        | 18.76 | 21600 | 0.3860          | 0.2415 |
| 0.0784        | 19.11 | 22000 | 0.4148          | 0.2454 |
| 0.0782        | 19.45 | 22400 | 0.3952          | 0.2471 |
| 0.0775        | 19.8  | 22800 | 0.3943          | 0.2434 |
| 0.0735        | 20.15 | 23200 | 0.4093          | 0.2405 |
| 0.0679        | 20.5  | 23600 | 0.3996          | 0.2362 |
| 0.0677        | 20.84 | 24000 | 0.4133          | 0.2365 |
| 0.0687        | 21.19 | 24400 | 0.4303          | 0.2330 |
| 0.0651        | 21.54 | 24800 | 0.4288          | 0.2326 |
| 0.0647        | 21.88 | 25200 | 0.4134          | 0.2347 |
| 0.0634        | 22.23 | 25600 | 0.4148          | 0.2312 |
| 0.0592        | 22.58 | 26000 | 0.4322          | 0.2315 |
| 0.06          | 22.93 | 26400 | 0.4050          | 0.2313 |
| 0.0561        | 23.27 | 26800 | 0.4260          | 0.2263 |
| 0.0546        | 23.62 | 27200 | 0.4228          | 0.2238 |
| 0.0548        | 23.97 | 27600 | 0.4140          | 0.2258 |
| 0.0505        | 24.32 | 28000 | 0.4304          | 0.2246 |
| 0.0501        | 24.66 | 28400 | 0.4241          | 0.2233 |
| 0.0481        | 25.01 | 28800 | 0.4385          | 0.2209 |
| 0.0469        | 25.36 | 29200 | 0.4451          | 0.2189 |
| 0.0464        | 25.71 | 29600 | 0.4397          | 0.2217 |
| 0.0438        | 26.05 | 30000 | 0.4419          | 0.2154 |
| 0.0432        | 26.4  | 30400 | 0.4366          | 0.2137 |
| 0.0419        | 26.75 | 30800 | 0.4371          | 0.2137 |
| 0.0419        | 27.1  | 31200 | 0.4552          | 0.2109 |
| 0.0392        | 27.44 | 31600 | 0.4496          | 0.2108 |
| 0.0386        | 27.79 | 32000 | 0.4585          | 0.2096 |
| 0.0387        | 28.14 | 32400 | 0.4496          | 0.2065 |
| 0.0367        | 28.48 | 32800 | 0.4646          | 0.2082 |
| 0.0357        | 28.83 | 33200 | 0.4553          | 0.2067 |
| 0.0355        | 29.18 | 33600 | 0.4615          | 0.2055 |
| 0.0345        | 29.53 | 34000 | 0.4670          | 0.2046 |
| 0.0346        | 29.87 | 34400 | 0.4658          | 0.2038 |


### Framework versions

- Transformers 4.29.0.dev0
- Pytorch 2.0.0+cu117
- Datasets 2.12.0
- Tokenizers 0.13.3