---
pipeline_tag: translation
license: "cc-by-nc-4.0"
inference: false
language:
- multilingual
- af
- am
- ar
- as
- ast
- ay
- az
- ba
- be
- bg
- bn
- br
- bs
- ca
- ceb
- cjk
- cs
- cy
- da
- de
- dyu
- el
- en
- es
- et
- fa
- ff
- fi
- fr
- fy
- ga
- gd
- gl
- gu
- ha
- he
- hi
- hr
- ht
- hu
- hy
- id
- ig
- ilo
- is
- it
- ja
- jv
- ka
- kac
- kam
- kea
- kg
- kk
- km
- kmb
- kmr
- kn
- ko
- ku
- ky
- lb
- lg
- ln
- lo
- lt
- luo
- lv
- mg
- mi
- mk
- ml
- mn
- mr
- ms
- mt
- my
- ne
- nl
- 'no'
- ns
- ny
- oc
- om
- or
- pa
- pl
- ps
- pt
- qu
- ro
- ru
- sd
- shn
- si
- sk
- sl
- sn
- so
- sq
- sr
- ss
- su
- sv
- sw
- ta
- te
- tg
- th
- ti
- tl
- tn
- tr
- uk
- umb
- ur
- uz
- vi
- wo
- xh
- yi
- yo
- zh
- zu
---

# Flores101: Large-Scale Multilingual Machine Translation

`flores101_mm100_175M` is a multilingual encoder-decoder (seq-to-seq) model trained for Many-to-Many multilingual translation. It was released in [this](https://github.com/facebookresearch/fairseq/tree/main/examples/flores101) repository.

The model architecture and config are the same as [M2M100](https://huggingface.co/facebook/m2m100_418M) implementation, but the **tokenizer should be modified** to adjust language codes.

**Demo**: https://huggingface.co/spaces/seyoungsong/flores101_mm100_175M

```python
from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer

hi_text = "जीवन एक चॉकलेट बॉक्स की तरह है।"
chinese_text = "生活就像一盒巧克力。"

model = M2M100ForConditionalGeneration.from_pretrained("seyoungsong/flores101_mm100_175M")
tokenizer: M2M100Tokenizer = M2M100Tokenizer.from_pretrained("seyoungsong/flores101_mm100_175M")

# FIX TOKENIZER!
tokenizer.lang_token_to_id = {t: i for t, i in zip(tokenizer.all_special_tokens, tokenizer.all_special_ids) if i > 5}
tokenizer.lang_code_to_token = {s.strip("_"): s for s in tokenizer.lang_token_to_id}
tokenizer.lang_code_to_id = {s.strip("_"): i for s, i in tokenizer.lang_token_to_id.items()}
tokenizer.id_to_lang_token = {i: s for s, i in tokenizer.lang_token_to_id.items()}

# translate Hindi to French
tokenizer.src_lang = "hi"
encoded_hi = tokenizer(hi_text, return_tensors="pt")
generated_tokens = model.generate(**encoded_hi, forced_bos_token_id=tokenizer.get_lang_id("fr"))
tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)
# => "La vie est comme une boîte de chocolat."

# translate Chinese to English
tokenizer.src_lang = "zh"
encoded_zh = tokenizer(chinese_text, return_tensors="pt")
generated_tokens = model.generate(**encoded_zh, forced_bos_token_id=tokenizer.get_lang_id("en"))
tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)
# => "Life is like a chocolate box."
```

## Languages covered

| Language         | lang code |
| ---------------- | --------- |
| Akrikaans        | af        |
| Amharic          | am        |
| Arabic           | ar        |
| Assamese         | as        |
| Asturian         | ast       |
| Aymara           | ay        |
| Azerbaijani      | az        |
| Bashkir          | ba        |
| Belarusian       | be        |
| Bulgarian        | bg        |
| Bengali          | bn        |
| Breton           | br        |
| Bosnian          | bs        |
| Catalan          | ca        |
| Cebuano          | ceb       |
| Chokwe           | cjk       |
| Czech            | cs        |
| Welsh            | cy        |
| Danish           | da        |
| German           | de        |
| Dyula            | dyu       |
| Greek            | el        |
| English          | en        |
| Spanish          | es        |
| Estonian         | et        |
| Persian          | fa        |
| Fulah            | ff        |
| Finnish          | fi        |
| French           | fr        |
| Western Frisian  | fy        |
| Irish            | ga        |
| Scottish Gaelic  | gd        |
| Galician         | gl        |
| Gujarati         | gu        |
| Hausa            | ha        |
| Hebrew           | he        |
| Hindi            | hi        |
| Croatian         | hr        |
| Haitian Creole   | ht        |
| Hungarian        | hu        |
| Armenian         | hy        |
| Indonesian       | id        |
| Igbo             | ig        |
| Iloko            | ilo       |
| Icelandic        | is        |
| Italian          | it        |
| Japanese         | ja        |
| Javanese         | jv        |
| Georgian         | ka        |
| Kachin           | kac       |
| Kamba            | kam       |
| Kabuverdianu     | kea       |
| Kongo            | kg        |
| Kazakh           | kk        |
| Central Khmer    | km        |
| Kimbundu         | kmb       |
| Northern Kurdish | kmr       |
| Kannada          | kn        |
| Korean           | ko        |
| Kurdish          | ku        |
| Kyrgyz           | ky        |
| Luxembourgish    | lb        |
| Ganda            | lg        |
| Lingala          | ln        |
| Lao              | lo        |
| Lithuanian       | lt        |
| Luo              | luo       |
| Latvian          | lv        |
| Malagasy         | mg        |
| Maori            | mi        |
| Macedonian       | mk        |
| Malayalam        | ml        |
| Mongolian        | mn        |
| Marathi          | mr        |
| Malay            | ms        |
| Maltese          | mt        |
| Burmese          | my        |
| Nepali           | ne        |
| Dutch            | nl        |
| Norwegian        | no        |
| Northern Sotho   | ns        |
| Nyanja           | ny        |
| Occitan          | oc        |
| Oromo            | om        |
| Oriya            | or        |
| Punjabi          | pa        |
| Polish           | pl        |
| Pashto           | ps        |
| Portuguese       | pt        |
| Quechua          | qu        |
| Romanian         | ro        |
| Russian          | ru        |
| Sindhi           | sd        |
| Shan             | shn       |
| Sinhala          | si        |
| Slovak           | sk        |
| Slovenian        | sl        |
| Shona            | sn        |
| Somali           | so        |
| Albanian         | sq        |
| Serbian          | sr        |
| Swati            | ss        |
| Sundanese        | su        |
| Swedish          | sv        |
| Swahili          | sw        |
| Tamil            | ta        |
| Telugu           | te        |
| Tajik            | tg        |
| Thai             | th        |
| Tigrinya         | ti        |
| Tagalog          | tl        |
| Tswana           | tn        |
| Turkish          | tr        |
| Ukrainian        | uk        |
| Umbundu          | umb       |
| Urdu             | ur        |
| Uzbek            | uz        |
| Vietnamese       | vi        |
| Wolof            | wo        |
| Xhosa            | xh        |
| Yiddish          | yi        |
| Yoruba           | yo        |
| Chinese          | zh        |
| Zulu             | zu        |