Agent Substrate: Advancing Kubernetes for Agents at Scale
Web page · https://developer.nvidia.com/blog/agent-substrate-advancing-kubernetes-for-agents-at-scale
AI agents spend most of their time idle, waiting on a human prompt, a tool result or a model response. Kubernetes wasn’t built for that. Running one Pod per...
First seen by Oriel on .
AI summary
NVIDIA presents Agent Substrate, a CNCF Sandbox runtime that multiplexes stateful, sandboxed AI agents across warm Kubernetes workers, claiming 10x higher density than standard containers while avoiding per-step Pod cold starts.
- Idle actors are checkpointed with memory and filesystem state, then restored into any free gVisor or microVM worker without invoking Kubernetes scheduling again.
- It targets coding assistants, bursty RL post-training environments, and short-lived GPU research sandboxes requiring persistent state, rapid startup, isolation, and fair resource sharing.
- Request parking waits boundedly for capacity, while microVM per-container CPU and memory limits prevent one container from exhausting an entire sandbox.
- Planned DRA-based device pluggability will initially support NVIDIA GPUs; GPU checkpoint restoration requires compatible hardware and may fall back to a cold start.
- Dynamo shares cache and GPU-pressure signals for agent-aware routing and suspend decisions, but APIs, GPU support, and parts of the integration remain experimental.
#kubernetes #ai agents #cncf #gpu scheduling
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| NVIDIA Developer Blog | https://developer.nvidia.com/blog/?p=124728 | https://developer.nvidia.com/blog/agent-substrate-advancing-kubernetes-for-agents-at-scale/ Read discussion |