Oriel — AI & technology worth noticing, every day.

Running Frontier-Class AI Models Across 8 NVIDIA DGX Spark Nodes

Web page · https://developer.nvidia.com/blog/running-frontier-class-ai-models-across-8-nvidia-dgx-spark-nodes

AI developers aren’t just running a single model anymore—they’re orchestrating multiple models in parallel, chaining inference calls, and experimenting...

First seen by Oriel on .

AI summary

NVIDIA describes eight-node DGX Spark clusters with 1 TB of unified memory for serving frontier-class models, routing workloads across model pools, and pre-training small models locally.

  • The full GLM 5.3 (753B) checkpoint, about 470 GB in NVFP4, runs across eight nodes; GLM 5.3 Flash generated roughly 80 tokens per second.
  • A Spark-optimized collective backend improves decode scaling on switch-connected clusters, but its extra data movement makes NCCL preferable as cluster size or concurrency grows.
  • Qwen3.8-27B decode throughput rises from 11.86 TPS on one node to 52.58 TPS on eight nodes using a switch and ROCEnante.
  • Switchyard routed requests between MiniCPM5 2B and Qwen3.8 Flash Next, matching Qwen-only quality while reaching about 1.5x to 3.0x aggregate throughput, depending on routing split.
  • Nanochat pre-training reached 7.79x speedup and 97.4% scaling efficiency across eight nodes; official eight-node NVIDIA Sync support was described as coming by year-end.

#nvidia dgx spark #llm inference #distributed training #model orchestration

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
NVIDIA Developer Bloghttps://developer.nvidia.com/blog/?p=123934https://developer.nvidia.com/blog/running-frontier-class-ai-models-across-8-nvidia-dgx-spark-nodes/ Read discussion