Oriel — AI & technology worth noticing, every day.

Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot

Web page · https://developer.nvidia.com/blog/start-and-restore-gpu-workloads-in-seconds-with-nvidia-snapshot

NVIDIA Snapshot, a new open-source component of the NVIDIA DSX AI Factory Platform, makes initialized GPU workloads reusable across inference...

First seen by Oriel on .

AI summary

NVIDIA Snapshot is an open-source Kubernetes checkpoint-and-restore system that reuses initialized GPU workloads, restoring inference replicas in seconds instead of minutes.

  • It captures a ready Pod’s CPU and GPU memory as an artifact and restores it on a node with the same GPU model and driver.
  • Kubernetes PodSnapshot and SnapshotJob resources create checkpoints; the nvidia.com/restore-from annotation starts a Pod from one, with a small workload pause-and-resume handshake.
  • Across vLLM, SGLang, and TensorRT LLM tests on NVIDIA B200, restores took 3.6–42.7 seconds and were 8x–36x faster than cold starts.
  • The tests included eight-GPU MoE checkpoints of about 600 GB to 1 TB; GPU state restoration took 18–29 seconds, while CRIU process restoration took 4.2–10.8 seconds.
  • Checkpoint creation took 15 seconds to 5.5 minutes, but checkpoint plus restore time was shorter than one cold start in every tested configuration; container image loading was excluded.

#kubernetes #gpu #llm inference #checkpointing

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
NVIDIA Developer Bloghttps://developer.nvidia.com/blog/?p=124736https://developer.nvidia.com/blog/start-and-restore-gpu-workloads-in-seconds-with-nvidia-snapshot/ Read discussion