Oriel — AI & technology worth noticing, every day.

Grace Blackwell needs a new scheduler: 27% more usable GPUs from the same racks

Web page · https://skypilot.ai/blog/grace-blackwell-needs-a-new-scheduler

First seen by Oriel on .

AI summary

The article argues that NVL72 clusters need rack-aware scheduling and failure recovery to realize NVLink performance, reporting 27% more usable capacity with 16-node slices and 14.4% higher training throughput from rack-local placement.

  • Tensor and expert parallelism rely on communication-intensive all-reduce and all-to-all operations; moving training off NVLink can reduce throughput by 43%.
  • NVL72 connects 72 GPUs through a non-blocking rack fabric, providing 1.8 TB/s per GPU and 130 TB/s aggregate internal bandwidth.
  • With roughly 2% node unavailability, a modeled 128-rack cluster provides 2,037 usable nodes with 16-node slices, versus 1,602 with full-rack slices.
  • SkyPilot’s Topology-Aware Scheduler keeps workers within racks, while Frontier Trainer replaces failed nodes with reserved spares and Automatic Defragmentation frees racks for blocked jobs.
  • In NVIDIA GR00T N1.7 pre-training across 512 GB300 GPUs, rack-aware placement cut iteration time from 1.59 s to 1.39 s; Auto Defrag only preempts workloads that can safely resume from checkpoints unless an admin opts in others.

#nvl72 #gpu scheduling #distributed training #nvlink

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hacker News: 144, latest observation ; +137 over observed period since

Comments

Comment count reported by the source.

  • Hacker News: 22, latest observation ; +22 over observed period since

Aliases

SourceExternal IDURL
Hacker News50290912https://skypilot.ai/blog/grace-blackwell-needs-a-new-scheduler Read discussion