metadata

license: cc-by-nc-4.0
language:
  - hyw
datasets:
  - mozilla-foundation/common_voice_16_1
  - google/fleurs
  - ReRooted
pipeline_tag: automatic-speech-recognition
tags:
  - speech-to-text
  - seamless_communication

SeamlessM4T v2 ASR for Western Armenian

This model is a fine-tuned version of the facebook/seamless-m4t-v2-large for the ASR task. Initially, it was fine-tuned on the Common Voice 16.1 and Google Fleurs datasets. Subsequently, it was further fine-tuned on the ReRooted corpus. The model achieves the following results on the test sets:

CV_wer: 0.308
CV_cer: 0.07
GF_wer: 0.311
GF_cer: 0.094

After fine-tuning on Western Armenian data, the model occasionally translates Eastern Armenian speech into Western Armenian(Colab link: Test SeamlessM4T v2 ASR for Western Armenian).

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-6
train_batch_size: 4
eval_batch_size: 1
seed: 43
optimizer: Adam with betas=(0.9, 0.98) and epsilon=1e-08
lr_scheduler_type: MyleLR
lr_scheduler_warmup_steps: 100

Framework versions

Pytorch 2.1.1
Fairseq2 0.2.0

Citation

For SeamlessM4T v2, please cite :

@inproceedings{seamless2023,
   title="Seamless: Multilingual Expressive and Streaming Speech Translation",
   author="{Seamless Communication}, Lo{\"i}c Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Mark Duppenthaler, Paul-Ambroise Duquenne, Brian Ellis, Hady Elsahar, Justin Haaheim, John Hoffman, Min-Jae Hwang, Hirofumi Inaguma, Christopher Klaiber, Ilia Kulikov, Pengwei Li, Daniel Licht, Jean Maillard, Ruslan Mavlyutov, Alice Rakotoarison, Kaushik Ram Sadagopan, Abinesh Ramakrishnan, Tuan Tran, Guillaume Wenzek, Yilin Yang, Ethan Ye, Ivan Evtimov, Pierre Fernandez, Cynthia Gao, Prangthip Hansanti, Elahe Kalbassi, Amanda Kallet, Artyom Kozhevnikov, Gabriel Mejia, Robin San Roman, Christophe Touret, Corinne Wong, Carleigh Wood, Bokai Yu, Pierre Andrews, Can Balioglu, Peng-Jen Chen, Marta R. Costa-juss{\`a}, Maha Elbayad, Hongyu Gong, Francisco Guzm{\'a}n, Kevin Heffernan, Somya Jain, Justine Kao, Ann Lee, Xutai Ma, Alex Mourachko, Benjamin Peloquin, Juan Pino, Sravya Popuri, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, Anna Sun, Paden Tomasello, Changhan Wang, Jeff Wang, Skyler Wang, Mary Williamson",
  journal={ArXiv},
  year={2023}
}