From the model card
Fields below are copied from the tags and counters on the HuggingFace repository FacebookAI/roberta-base at our last fetch. They are set by the uploader, not verified by us; rows with no tag are omitted. How this page is made.
- Publisher (HF namespace)
- FacebookAI
- Pipeline tag
- fill-mask
- Library
- Transformers
- Framework tags
- PyTorch, TensorFlow, JAX, Rust (candle)
- Weight formats
- safetensors
- License tag
mit— read the license file in the repo before relying on it- Language tags
- English (en)
- Papers cited
- arXiv:1907.11692, arXiv:1806.02847
- Datasets declared
- bookcorpus, wikipedia
- Downloads (HF counter at last fetch)
- 9,585,589
- Likes (HF counter at last fetch)
- 640
- Model card
- https://huggingface.co/FacebookAI/roberta-base
Use cases
- Fine-tuning for text classification (sentiment analysis, topic detection, intent recognition)
- Named entity recognition with a token classification head
- Natural language inference and textual entailment
- Extractive question answering with span prediction
- Sentence encoding as a higher-quality alternative to original BERT
Pros
- More rigorous pre-training than BERT yields better NLU task performance
- Multi-framework support (PyTorch, TF, JAX, Rust, safetensors)
- MIT license; large ecosystem of fine-tuned domain-specific variants
- Well-understood behavior from extensive published NLP research
Cons
- English-only; no multilingual variant in this checkpoint
- 512-token context limit requires chunking for long documents
- Encoder-only architecture cannot generate free-form text
- Surpassed on most benchmarks by DeBERTa variants and more recent efficient encoders
- Heavier than distilled alternatives for limited accuracy gains on easy tasks
Tags
transformerspytorchtfjaxrustsafetensorsrobertafill-maskexbertendataset:bookcorpusdataset:wikipediaarxiv:1907.11692arxiv:1806.02847license:mitendpoints_compatibleregion:usdeploy:sagemakerdeploy:azure