nvidia/GLM-5.3-Flash-FP4-Local
AI model · https://huggingface.co/nvidia/GLM-5.3-Flash-FP4-Local
First seen by Oriel on .
AI summary
NVIDIA's GLM-5.3-Flash-FP4-Local is an NVFP4 quantized, multimodal 320B-parameter MoE model for local inference, requiring specialized community vLLM containers until upstream support arrives.
- The model activates 18B parameters, accepts text, images, and video, and supports context lengths up to 1M tokens.
- It uses quantization-aware distillation from a BF16 teacher; benchmark scores are 91.3% on GPQA Diamond and 92.4% on ToolEvalBench, versus 92.1% and 91.9% for BF16.
- The checkpoint uses approximately 166 GiB of VRAM and was validated on 2 x NVIDIA RTX PRO 6000 Blackwell or 2 x NVIDIA DGX Spark 128 GB.
- Its NVFP4 scale factors use a new lossless compression format not yet supported by upstream vLLM, so deployment requires the provided community containers.
- The model is intended for developers deploying agents, chatbots, RAG systems, and other AI applications; use is governed by the NVIDIA Open Model Agreement, while the base model uses the MIT License.
#large language models #quantization #vllm #multimodal ai
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Likes
Total likes reported by the source.
- Hugging Face Models: 118, latest observation ; +12 over observed period since
Downloads (30d)
Rolling 30-day downloads. A change in this value is not new downloads during the observed period.
- Hugging Face Models: 4,243, latest observation ; +1,301 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Models | nvidia/GLM-5.3-Flash-FP4-Local | https://huggingface.co/nvidia/GLM-5.3-Flash-FP4-Local Read discussion |