AI Tools.

Search
fill mask by FacebookAI

xlm-roberta-large

XLM-RoBERTa Large, the 560-million-parameter multilingual encoder from Facebook AI, trained on 2.5TB of CommonCrawl data across 100 languages. It offers stronger multilingual language understanding than the base variant across classification, NER, and cross-lingual tasks, at roughly 4x the compute cost. MIT licensed with multi-framework support.

Summary text generated by an automated pipeline from the model card · Not individually reviewed or run by us · How this page is made

Share: X Reddit HN LinkedIn

From the model card

Fields below are copied from the tags and counters on the HuggingFace repository FacebookAI/xlm-roberta-large at our last fetch. They are set by the uploader, not verified by us; rows with no tag are omitted. How this page is made.

Publisher (HF namespace)
FacebookAI
Pipeline tag
fill-mask
Library
Transformers
Framework tags
PyTorch, TensorFlow, JAX
Weight formats
ONNX, safetensors
License tag
mit — read the license file in the repo before relying on it
Language tags
multilingual; Afrikaans (af), Amharic (am), Arabic (ar), Assamese (as), Azerbaijani (az), Belarusian (be), Bulgarian (bg), Bangla (bn), Breton (br), Bosnian (bs), Catalan (ca), Czech (cs), Welsh (cy), Danish (da), German (de), Greek (el), English (en), Esperanto (eo), Spanish (es), Estonian (et), Basque (eu), Persian (fa), Finnish (fi), French (fr), Western Frisian (fy), Irish (ga), Scottish Gaelic (gd), Galician (gl), Gujarati (gu), Hausa (ha), Hebrew (he), Hindi (hi), Croatian (hr), Hungarian (hu), Armenian (hy), Indonesian (id), Icelandic (is), Italian (it), Japanese (ja), Javanese (jv), Georgian (ka), Kazakh (kk), Khmer (km), Kannada (kn), Korean (ko), Kurdish (ku), Kyrgyz (ky), Latin (la), Lao (lo), Lithuanian (lt), Latvian (lv), Malagasy (mg), Macedonian (mk), Malayalam (ml), Mongolian (mn), Marathi (mr), Malay (ms), Burmese (my), Nepali (ne), Dutch (nl), Norwegian (no), Oromo (om), Odia (or), Punjabi (pa), Polish (pl), Pashto (ps), Portuguese (pt), Romanian (ro), Russian (ru), Sanskrit (sa), Sindhi (sd), Sinhala (si), Slovak (sk), Slovenian (sl), Somali (so), Albanian (sq), Serbian (sr), Sundanese (su), Swedish (sv), Swahili (sw), Tamil (ta), Telugu (te), Thai (th), Filipino (tl), Turkish (tr), Uyghur (ug), Ukrainian (uk), Urdu (ur), Uzbek (uz), Vietnamese (vi), Xhosa (xh), Yiddish (yi), Chinese (zh)
Papers cited
arXiv:1911.02116
Downloads (HF counter at last fetch)
3,801,865
Likes (HF counter at last fetch)
527
Model card
https://huggingface.co/FacebookAI/xlm-roberta-large

Use cases

  • High-accuracy multilingual NER and sequence labeling
  • Cross-lingual text classification requiring strong encoder quality
  • Multilingual natural language inference at research quality
  • Sentence embedding for 100-language corpora when accuracy matters more than speed
  • Foundation for multilingual fine-tuned classifiers in production

Pros

  • 560M parameters provide stronger multilingual representations than base
  • MIT license; multi-framework support (PyTorch, TF, JAX, ONNX, safetensors)
  • Widely published cross-lingual benchmark results (XNLI, WikiANN)
  • 100-language coverage from large-scale CommonCrawl training

Cons

  • 4x compute cost vs. XLM-RoBERTa-base for marginal multilingual gains on simpler tasks
  • High-resource languages still outperformed by dedicated monolingual models
  • 512-token context limit for long-document tasks
  • Not suitable for text generation
  • Encoder-only architecture limits use cases vs. modern multilingual LLMs

Tags

transformerspytorchtfjaxonnxsafetensorsxlm-robertafill-maskexbertmultilingualafamarasazbebgbnbrbs