Oriel — AI & technology worth noticing, every day.

NVIDIA Vera Rubin NVL72 Leads Agentic Inference Performance on MLPerf Endpoints v1.0

Web page · https://developer.nvidia.com/blog/nvidia-vera-rubin-nvl72-leads-agentic-inference-performance-on-mlperf-endpoints-v1-0

Agentic workloads are demanding more from inference systems. AI agents carry growing conversation histories across turns and generate a mix of reasoning, tool...

First seen by Oriel on .

AI summary

NVIDIA reports provisional MLPerf Endpoints v1.0 results showing Vera Rubin NVL72 delivers substantially higher agentic inference throughput per kilowatt than GB300 NVL72 at comparable responsiveness.

  • For Kimi K3, Vera Rubin achieved 41,414.2 system tokens per second at 1,024-query concurrency and 50.4 tokens per second per user.
  • Compared with GB300 NVL72, reported throughput-per-kilowatt ratios were 10x for Kimi K3, 9x for DeepSeek-V4.1-Flash, and 5x for Qwen3.6-35B-A3B.
  • The workloads model multi-turn coding and enterprise workflows with growing context, tool calls, shared instructions, and business rules.
  • Prefix caching, state reuse, NVIDIA Dynamo routing, disaggregated prefill and decode, model-specific kernels, and speculative verification reduce redundant computation and execution overhead.
  • Vera Rubin results are provisional and pending MLCommons Association verification, while the cited GB300 results are verified.

#agentic inference #mlperf #nvidia vera rubin #gpu serving

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
NVIDIA Developer Bloghttps://developer.nvidia.com/blog/?p=125279https://developer.nvidia.com/blog/nvidia-vera-rubin-nvl72-leads-agentic-inference-performance-on-mlperf-endpoints-v1-0/ Read discussion