Oriel — AI & technology worth noticing, every day.

nvidia/GLM-5.3-Flash-FP4-Local

AI model · https://huggingface.co/nvidia/GLM-5.3-Flash-FP4-Local

First seen by Oriel on .

AI summary

NVIDIA's GLM-5.3-Flash-FP4-Local is an NVFP4 quantized, multimodal 320B-parameter MoE model for local inference, requiring specialized community vLLM containers until upstream support arrives.

  • The model activates 18B parameters, accepts text, images, and video, and supports context lengths up to 1M tokens.
  • It uses quantization-aware distillation from a BF16 teacher; benchmark scores are 91.3% on GPQA Diamond and 92.4% on ToolEvalBench, versus 92.1% and 91.9% for BF16.
  • The checkpoint uses approximately 166 GiB of VRAM and was validated on 2 x NVIDIA RTX PRO 6000 Blackwell or 2 x NVIDIA DGX Spark 128 GB.
  • Its NVFP4 scale factors use a new lossless compression format not yet supported by upstream vLLM, so deployment requires the provided community containers.
  • The model is intended for developers deploying agents, chatbots, RAG systems, and other AI applications; use is governed by the NVIDIA Open Model Agreement, while the base model uses the MIT License.

#large language models #quantization #vllm #multimodal ai

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Likes

Total likes reported by the source.

  • Hugging Face Models: 118, latest observation ; +12 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Models: 4,243, latest observation ; +1,301 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Modelsnvidia/GLM-5.3-Flash-FP4-Localhttps://huggingface.co/nvidia/GLM-5.3-Flash-FP4-Local Read discussion