AI Tools.

Search
fill mask by FacebookAI

xlm-roberta-base

XLM-RoBERTa base from Facebook AI, pre-trained on 2.5TB of filtered CommonCrawl text across 100 languages using the RoBERTa training procedure. Enables cross-lingual transfer — models fine-tuned on labeled English data can infer on other languages without parallel annotations. The standard starting point for multilingual classification and token-level tasks.

Summary text generated by an automated pipeline from the model card · Not individually reviewed or run by us · How this page is made

Share: X Reddit HN LinkedIn

From the model card

Fields below are copied from the tags and counters on the HuggingFace repository FacebookAI/xlm-roberta-base at our last fetch. They are set by the uploader, not verified by us; rows with no tag are omitted. How this page is made.

Publisher (HF namespace)
FacebookAI
Pipeline tag
fill-mask
Library
Transformers
Framework tags
PyTorch, TensorFlow, JAX
Weight formats
ONNX, safetensors
License tag
mit — read the license file in the repo before relying on it
Language tags
multilingual; Afrikaans (af), Amharic (am), Arabic (ar), Assamese (as), Azerbaijani (az), Belarusian (be), Bulgarian (bg), Bangla (bn), Breton (br), Bosnian (bs), Catalan (ca), Czech (cs), Welsh (cy), Danish (da), German (de), Greek (el), English (en), Esperanto (eo), Spanish (es), Estonian (et), Basque (eu), Persian (fa), Finnish (fi), French (fr), Western Frisian (fy), Irish (ga), Scottish Gaelic (gd), Galician (gl), Gujarati (gu), Hausa (ha), Hebrew (he), Hindi (hi), Croatian (hr), Hungarian (hu), Armenian (hy), Indonesian (id), Icelandic (is), Italian (it), Japanese (ja), Javanese (jv), Georgian (ka), Kazakh (kk), Khmer (km), Kannada (kn), Korean (ko), Kurdish (ku), Kyrgyz (ky), Latin (la), Lao (lo), Lithuanian (lt), Latvian (lv), Malagasy (mg), Macedonian (mk), Malayalam (ml), Mongolian (mn), Marathi (mr), Malay (ms), Burmese (my), Nepali (ne), Dutch (nl), Norwegian (no), Oromo (om), Odia (or), Punjabi (pa), Polish (pl), Pashto (ps), Portuguese (pt), Romanian (ro), Russian (ru), Sanskrit (sa), Sindhi (sd), Sinhala (si), Slovak (sk), Slovenian (sl), Somali (so), Albanian (sq), Serbian (sr), Sundanese (su), Swedish (sv), Swahili (sw), Tamil (ta), Telugu (te), Thai (th), Filipino (tl), Turkish (tr), Uyghur (ug), Ukrainian (uk), Urdu (ur), Uzbek (uz), Vietnamese (vi), Xhosa (xh), Yiddish (yi), Chinese (zh)
Papers cited
arXiv:1911.02116
Downloads (HF counter at last fetch)
21,062,407
Likes (HF counter at last fetch)
893
Model card
https://huggingface.co/FacebookAI/xlm-roberta-base

Use cases

  • Multilingual NER without separate per-language models
  • Cross-lingual text classification (train in English, infer in other languages)
  • Multilingual sentiment analysis across international product reviews
  • Sequence labeling on low-resource languages via cross-lingual transfer
  • Universal sentence encoding for 100-language document corpora

Pros

  • 100-language coverage in a single model checkpoint
  • RoBERTa training rigor applied multilingually yields strong cross-lingual transfer
  • Multi-framework support (PyTorch, TF, JAX, ONNX, Rust); MIT license
  • Strong performance on XNLI and WikiANN multilingual benchmarks

Cons

  • Shared multilingual vocabulary degrades per-language token efficiency vs. monolingual models
  • Outperformed by dedicated monolingual models on high-resource languages
  • 512-token context limit
  • High-resource languages (English, German, French) dominate training data
  • Base size limits accuracy on tasks requiring deep language reasoning

Tags

transformerspytorchtfjaxonnxsafetensorsxlm-robertafill-maskexbertmultilingualafamarasazbebgbnbrbs