metadata

library_name: transformers
license: cc-by-nc-4.0
base_model: facebook/mms-1b-all
tags:
  - automatic-speech-recognition
  - bigcgen
  - mms
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: mms-1b-bigcgen-combined-20hrs-model
    results: []

mms-1b-bigcgen-combined-20hrs-model

This model is a fine-tuned version of facebook/mms-1b-all on the BIGCGEN - BEM dataset. It achieves the following results on the evaluation set:

Loss: inf
Wer: 0.5166

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 4
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 8
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 100
training_steps: 2500
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
14.8448	0.0762	100	inf	1.0079
6.2506	0.1524	200	inf	1.0042
5.5314	0.2287	300	inf	1.0270
3.4418	0.3049	400	inf	0.5906
1.9396	0.3811	500	inf	0.5762
1.698	0.4573	600	inf	0.5566
1.5483	0.5335	700	inf	0.5571
1.6501	0.6098	800	inf	0.5487
1.5528	0.6860	900	inf	0.5471
1.5398	0.7622	1000	inf	0.5479
1.6413	0.8384	1100	inf	0.5304
1.418	0.9146	1200	inf	0.5283
1.5625	0.9909	1300	inf	0.5265
1.4753	1.0671	1400	inf	0.5347
1.616	1.1433	1500	inf	0.5309
1.3802	1.2195	1600	inf	0.5246
1.4105	1.2957	1700	inf	0.5197
1.3793	1.3720	1800	inf	0.5288
1.3991	1.4482	1900	inf	0.5140
1.5838	1.5244	2000	inf	0.5239
1.6283	1.6006	2100	inf	0.5144
1.4131	1.6768	2200	inf	0.5135
1.388	1.7530	2300	inf	0.5137
1.3846	1.8293	2400	inf	0.5145
1.497	1.9055	2500	inf	0.5167

Framework versions

Transformers 4.47.1
Pytorch 2.5.1+cu124
Datasets 3.2.0
Tokenizers 0.21.0