From the model card
Fields below are copied from the tags and counters on the HuggingFace repository Qwen/Qwen3-32B-FP8 at our last fetch. They are set by the uploader, not verified by us; rows with no tag are omitted. How this page is made.
- Publisher (HF namespace)
- Qwen
- Pipeline tag
- text-generation
- Library
- Transformers
- Weight formats
- safetensors
- License tag
apache-2.0— read the license file in the repo before relying on it- Lineage
-
- base model Qwen/Qwen3-32B
- quantized from Qwen/Qwen3-32B
- Papers cited
- arXiv:2309.00071, arXiv:2505.09388
- Downloads (HF counter at last fetch)
- 389,347
- Likes (HF counter at last fetch)
- 83
- Model card
- https://huggingface.co/Qwen/Qwen3-32B-FP8
Use cases
- High-throughput Qwen3-32B serving on H100 GPUs
- Cost-efficient inference where full bf16 is too expensive
- Qwen3 capability at reduced VRAM cost in multi-tenant deployments
- Benchmarking FP8 accuracy degradation on Qwen3-32B tasks
Pros
- Apache-2.0 license
- FP8 is an official Alibaba quantization — not community-derived
- ~50% VRAM reduction vs bf16 on H100 hardware
- Transformers and TGI compatible
Cons
- FP8 requires Hopper (H100) or newer GPUs — no benefit on A100 or consumer cards
- FP8 quantization introduces small but measurable accuracy regression on some tasks
- 20 GB+ VRAM still required even at FP8 — not entry-level hardware
- Less established tooling than INT4 quantization formats
Tags
transformerssafetensorsqwen3text-generationconversationalarxiv:2309.00071arxiv:2505.09388base_model:Qwen/Qwen3-32Bbase_model:quantized:Qwen/Qwen3-32Blicense:apache-2.0text-generation-inferenceendpoints_compatiblefp8region:us