---
library_name: transformers
language:
- bem
license: mit
base_model: facebook/w2v-bert-2.0
tags:
- generated_from_trainer
datasets:
- BIG_C/BEMBA
metrics:
- wer
model-index:
- name: facebook/w2v-bert-2.0
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: BIG_C
      type: BIG_C/BEMBA
    metrics:
    - name: Wer
      type: wer
      value: 0.4003345055322069
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# facebook/w2v-bert-2.0

This model is a fine-tuned version of [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0) on the BIG_C dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4054
- Wer: 0.4003
- Cer: 0.0766

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 2
- eval_batch_size: 1
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.025
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step   | Validation Loss | Wer    | Cer    |
|:-------------:|:-----:|:------:|:---------------:|:------:|:------:|
| 1.0933        | 1.0   | 41178  | 0.5653          | 0.4240 | 0.1120 |
| 0.5359        | 2.0   | 82356  | 0.5139          | 0.3772 | 0.1026 |
| 0.4943        | 3.0   | 123534 | 0.4832          | 0.3560 | 0.0996 |
| 0.4599        | 4.0   | 164712 | 0.4774          | 0.3378 | 0.0948 |
| 0.4331        | 5.0   | 205890 | 0.4882          | 0.3305 | 0.0931 |
| 0.4092        | 6.0   | 247068 | 0.4580          | 0.3281 | 0.0921 |
| 0.3826        | 7.0   | 288246 | 0.4873          | 0.3232 | 0.0903 |
| 0.3536        | 8.0   | 329424 | 0.5067          | 0.3227 | 0.0908 |
| 0.3231        | 9.0   | 370602 | 0.5101          | 0.3274 | 0.0938 |
| 0.2924        | 10.0  | 411780 | 0.5481          | 0.3290 | 0.0927 |
| 0.263         | 11.0  | 452958 | 0.5684          | 0.3320 | 0.0927 |
| 0.2364        | 12.0  | 494136 | 0.5973          | 0.3362 | 0.0935 |
| 0.2135        | 13.0  | 535314 | 0.6344          | 0.3405 | 0.0951 |
| 0.1941        | 14.0  | 576492 | 0.7075          | 0.3370 | 0.0939 |
| 0.1765        | 15.0  | 617670 | 0.7800          | 0.3398 | 0.0947 |
| 0.1615        | 16.0  | 658848 | 0.8164          | 0.3389 | 0.0941 |
| 0.1482        | 17.0  | 700026 | 0.8562          | 0.3410 | 0.0949 |


### Framework versions

- Transformers 4.45.2
- Pytorch 2.2.0+cu121
- Datasets 3.0.1
- Tokenizers 0.20.1