From the model card
Fields below are copied from the tags and counters on the HuggingFace repository distilbert/distilbert-base-uncased at our last fetch. They are set by the uploader, not verified by us; rows with no tag are omitted. How this page is made.
- Publisher (HF namespace)
- distilbert
- Pipeline tag
- fill-mask
- Library
- Transformers
- Framework tags
- PyTorch, TensorFlow, JAX, Rust (candle)
- Weight formats
- safetensors
- License tag
apache-2.0— read the license file in the repo before relying on it- Language tags
- English (en)
- Papers cited
- arXiv:1910.01108
- Datasets declared
- bookcorpus, wikipedia
- Downloads (HF counter at last fetch)
- 6,761,868
- Likes (HF counter at last fetch)
- 1,094
- Model card
- https://huggingface.co/distilbert/distilbert-base-uncased
Use cases
- Text classification in latency-constrained environments (sentiment, intent)
- NER where BERT-level performance is needed at lower compute cost
- Extractive QA on shorter passages with faster inference requirement
- Edge deployment where BERT-base is too large
- High-throughput classification pipelines where latency per request matters
Pros
- 40% smaller and 60% faster than BERT-base with ~97% performance retained
- Multi-framework support (PyTorch, TF, JAX, Rust, ONNX, safetensors)
- Apache 2.0 license; large ecosystem of fine-tuned checkpoints
- Lowercase tokenization consistent with BERT-base-uncased fine-tuned models
Cons
- Performance gap vs. BERT-base grows on more complex NLU tasks
- Lowercase tokenization cannot distinguish case — limits NER on proper nouns
- 512-token context limit
- Encoder-only; cannot generate text
- Surpassed by more efficient distilled models (MiniLM, TinyBERT) on the speed-accuracy frontier
Tags
transformerspytorchtfjaxrustsafetensorsdistilbertfill-maskexbertendataset:bookcorpusdataset:wikipediaarxiv:1910.01108license:apache-2.0endpoints_compatibleregion:usdeploy:sagemakerdeploy:azure