---
library_name: transformers
license: cc-by-nc-4.0
base_model: facebook/mms-1b-all
tags:
- automatic-speech-recognition
- bigcgen
- mms
- generated_from_trainer
metrics:
- wer
model-index:
- name: mms-1b-bigcgen-combined-10hrs-model
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# mms-1b-bigcgen-combined-10hrs-model

This model is a fine-tuned version of [facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all) on the BIGCGEN - BEM dataset.
It achieves the following results on the evaluation set:
- Loss: inf
- Wer: 0.5167

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 8
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- training_steps: 2500
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Wer    |
|:-------------:|:------:|:----:|:---------------:|:------:|
| 14.705        | 0.1528 | 100  | inf             | 1.0004 |
| 6.0879        | 0.3056 | 200  | inf             | 1.0203 |
| 5.4042        | 0.4584 | 300  | inf             | 1.0160 |
| 2.8156        | 0.6112 | 400  | inf             | 0.6058 |
| 1.9111        | 0.7639 | 500  | inf             | 0.5724 |
| 1.82          | 0.9167 | 600  | inf             | 0.5643 |
| 1.6294        | 1.0688 | 700  | inf             | 0.5685 |
| 1.6856        | 1.2215 | 800  | inf             | 0.5530 |
| 1.6363        | 1.3743 | 900  | inf             | 0.5501 |
| 1.5114        | 1.5271 | 1000 | inf             | 0.5417 |
| 1.5417        | 1.6799 | 1100 | inf             | 0.5358 |
| 1.6518        | 1.8327 | 1200 | inf             | 0.5337 |
| 1.4795        | 1.9855 | 1300 | inf             | 0.5292 |
| 1.5822        | 2.1375 | 1400 | inf             | 0.5278 |
| 1.4938        | 2.2903 | 1500 | inf             | 0.5194 |
| 1.5701        | 2.4431 | 1600 | inf             | 0.5353 |
| 1.47          | 2.5959 | 1700 | inf             | 0.5183 |
| 1.4109        | 2.7487 | 1800 | inf             | 0.5342 |
| 1.3993        | 2.9015 | 1900 | inf             | 0.5175 |
| 1.4848        | 3.0535 | 2000 | inf             | 0.5177 |
| 1.4331        | 3.2063 | 2100 | inf             | 0.5286 |
| 1.4392        | 3.3591 | 2200 | inf             | 0.5213 |
| 1.3324        | 3.5118 | 2300 | inf             | 0.5161 |
| 1.501         | 3.6646 | 2400 | inf             | 0.5160 |
| 1.3526        | 3.8174 | 2500 | inf             | 0.5163 |


### Framework versions

- Transformers 4.47.1
- Pytorch 2.5.1+cu124
- Datasets 3.2.0
- Tokenizers 0.21.0