NVIDIA Vera Rubin NVL72 Leads Agentic Inference Performance on MLPerf Endpoints v1.0
Agentic workloads are demanding more from inference systems. AI agents carry growing conversation histories across turns and generate a mix of reasoning, tool...
First seen by Oriel on .
AI summary
NVIDIA reports provisional MLPerf Endpoints v1.0 results showing Vera Rubin NVL72 delivers substantially higher agentic inference throughput per kilowatt than GB300 NVL72 at comparable responsiveness.
- For Kimi K3, Vera Rubin achieved 41,414.2 system tokens per second at 1,024-query concurrency and 50.4 tokens per second per user.
- Compared with GB300 NVL72, reported throughput-per-kilowatt ratios were 10x for Kimi K3, 9x for DeepSeek-V4.1-Flash, and 5x for Qwen3.6-35B-A3B.
- The workloads model multi-turn coding and enterprise workflows with growing context, tool calls, shared instructions, and business rules.
- Prefix caching, state reuse, NVIDIA Dynamo routing, disaggregated prefill and decode, model-specific kernels, and speculative verification reduce redundant computation and execution overhead.
- Vera Rubin results are provisional and pending MLCommons Association verification, while the cited GB300 results are verified.
#agentic inference #mlperf #nvidia vera rubin #gpu serving
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| NVIDIA Developer Blog | https://developer.nvidia.com/blog/?p=125279 | https://developer.nvidia.com/blog/nvidia-vera-rubin-nvl72-leads-agentic-inference-performance-on-mlperf-endpoints-v1-0/ Read discussion |