From the model card
Fields below are copied from the tags and counters on the HuggingFace repository facebook/fasttext-language-identification at our last fetch. They are set by the uploader, not verified by us; rows with no tag are omitted. How this page is made.
- Publisher (HF namespace)
- Pipeline tag
- text-classification
- License tag
cc-by-nc-4.0— read the license file in the repo before relying on it- Papers cited
- arXiv:1607.04606, arXiv:1802.06893, arXiv:1607.01759, arXiv:1612.03651
- Downloads (HF counter at last fetch)
- 427,059
- Likes (HF counter at last fetch)
- 278
- Model card
- https://huggingface.co/facebook/fasttext-language-identification
Use cases
- Language detection preprocessing in multilingual NLP pipelines
- Filtering multilingual corpora by language label
- Language routing for translation or ASR system selection
- Content moderation to detect unexpected languages in user input
Pros
- 176 language coverage is broad — handles most real-world language identification needs
- Extremely fast: thousands of predictions per second on CPU
- Tiny model footprint (<1 MB)
- Well-tested in production across many organizations
Cons
- Short text accuracy degrades significantly — fails on single words or very short phrases
- Code-switching text may produce unreliable results
- Some language pairs (e.g. Malay/Indonesian, Serbian/Croatian/Bosnian) are confused at higher rates
- fastText format requires the fastText Python library, not standard transformers
Tags
fasttexttext-classificationlanguage-identificationarxiv:1607.04606arxiv:1802.06893arxiv:1607.01759arxiv:1612.03651license:cc-by-nc-4.0region:us