Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot
Web page · https://developer.nvidia.com/blog/start-and-restore-gpu-workloads-in-seconds-with-nvidia-snapshot
NVIDIA Snapshot, a new open-source component of the NVIDIA DSX AI Factory Platform, makes initialized GPU workloads reusable across inference...
First seen by Oriel on .
AI summary
NVIDIA Snapshot is an open-source Kubernetes checkpoint-and-restore system that reuses initialized GPU workloads, restoring inference replicas in seconds instead of minutes.
- It captures a ready Pod’s CPU and GPU memory as an artifact and restores it on a node with the same GPU model and driver.
- Kubernetes PodSnapshot and SnapshotJob resources create checkpoints; the nvidia.com/restore-from annotation starts a Pod from one, with a small workload pause-and-resume handshake.
- Across vLLM, SGLang, and TensorRT LLM tests on NVIDIA B200, restores took 3.6–42.7 seconds and were 8x–36x faster than cold starts.
- The tests included eight-GPU MoE checkpoints of about 600 GB to 1 TB; GPU state restoration took 18–29 seconds, while CRIU process restoration took 4.2–10.8 seconds.
- Checkpoint creation took 15 seconds to 5.5 minutes, but checkpoint plus restore time was shorter than one cold start in every tested configuration; container image loading was excluded.
#kubernetes #gpu #llm inference #checkpointing
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| NVIDIA Developer Blog | https://developer.nvidia.com/blog/?p=124736 | https://developer.nvidia.com/blog/start-and-restore-gpu-workloads-in-seconds-with-nvidia-snapshot/ Read discussion |