From the model card
Fields below are copied from the tags and counters on the HuggingFace repository QuantTrio/DeepSeek-V3.2-AWQ at our last fetch. They are set by the uploader, not verified by us; rows with no tag are omitted. How this page is made.
- Publisher (HF namespace)
- QuantTrio
- Pipeline tag
- text-generation
- Library
- Transformers, vLLM
- Weight formats
- safetensors
- License tag
mit— read the license file in the repo before relying on it- Lineage
-
- base model deepseek-ai/DeepSeek-V3.2
- quantized from deepseek-ai/DeepSeek-V3.2
- Downloads (HF counter at last fetch)
- 437,841
- Likes (HF counter at last fetch)
- 11
- Model card
- https://huggingface.co/QuantTrio/DeepSeek-V3.2-AWQ
Use cases
- Deploying DeepSeek V3.2 on GPU clusters with limited per-card VRAM
- High-throughput vLLM serving with reduced memory budget
- Cost-efficient inference for long-context enterprise tasks
- Comparing AWQ quality against GPTQ or GGUF alternatives
- Serving chat endpoints backed by a frontier-class model at lower hardware cost
Pros
- AWQ typically outperforms GPTQ at equivalent bit-width in perplexity
- MIT license on this quantised variant
- vLLM-native format for high-throughput batching
- 4-bit reduces VRAM roughly 4x vs BF16
Cons
- AWQ dequantisation adds per-token overhead; latency increases vs BF16 on fast hardware
- Cannot be fine-tuned further after AWQ quantisation
- Community quantisation; no guarantee of bit-for-bit reproducibility with official weights
- Requires vLLM-compatible GPU; consumer GPUs may hit memory limits even quantised
Tags
transformerssafetensorsdeepseek_v32text-generationvLLMAWQconversationalbase_model:deepseek-ai/DeepSeek-V3.2base_model:quantized:deepseek-ai/DeepSeek-V3.2license:mitendpoints_compatible4-bitawqregion:us