From the model card
Fields below are copied from the tags and counters on the HuggingFace repository facebook/opt-1.3b at our last fetch. They are set by the uploader, not verified by us; rows with no tag are omitted. How this page is made.
- Publisher (HF namespace)
- Pipeline tag
- text-generation
- Library
- Transformers
- Framework tags
- PyTorch, TensorFlow, JAX
- License tag
other— read the license file in the repo before relying on it- Language tags
- English (en)
- Papers cited
- arXiv:2205.01068, arXiv:2005.14165
- Downloads (HF counter at last fetch)
- 352,352
- Likes (HF counter at last fetch)
- 184
- Model card
- https://huggingface.co/facebook/opt-1.3b
Use cases
- Reproducing OPT-paper ablations on architecture and data choices
- Low-memory text generation for researchers without large GPU clusters
- Comparative studies on pre-2023 LLM training practices
- Teaching language model concepts with a well-documented checkpoint
- Baseline in continual learning and catastrophic forgetting experiments
Pros
- Full training logs and logbook released alongside weights for transparency
- PyTorch, TF, and JAX weights available
- 1.3B parameters run feasibly on 4GB VRAM or CPU
- Extensively benchmarked in the literature for easy comparison
Cons
- Significantly outperformed by Llama 2 7B and its descendants on every benchmark
- 'Other' license with Meta's terms limits some commercial deployments
- No instruction tuning; continuation model with poor instruction following
- Training data biases are documented in the OPT paper and carry real risks
Tags
transformerspytorchtfjaxtext-generationoptenarxiv:2205.01068arxiv:2005.14165license:otherdeploy:azureregion:us