Oriel — AI & technology worth noticing, every day.

Agent Substrate: Advancing Kubernetes for Agents at Scale

Web page · https://developer.nvidia.com/blog/agent-substrate-advancing-kubernetes-for-agents-at-scale

AI agents spend most of their time idle, waiting on a human prompt, a tool result or a model response. Kubernetes wasn’t built for that. Running one Pod per...

First seen by Oriel on .

AI summary

NVIDIA presents Agent Substrate, a CNCF Sandbox runtime that multiplexes stateful, sandboxed AI agents across warm Kubernetes workers, claiming 10x higher density than standard containers while avoiding per-step Pod cold starts.

  • Idle actors are checkpointed with memory and filesystem state, then restored into any free gVisor or microVM worker without invoking Kubernetes scheduling again.
  • It targets coding assistants, bursty RL post-training environments, and short-lived GPU research sandboxes requiring persistent state, rapid startup, isolation, and fair resource sharing.
  • Request parking waits boundedly for capacity, while microVM per-container CPU and memory limits prevent one container from exhausting an entire sandbox.
  • Planned DRA-based device pluggability will initially support NVIDIA GPUs; GPU checkpoint restoration requires compatible hardware and may fall back to a cold start.
  • Dynamo shares cache and GPU-pressure signals for agent-aware routing and suspend decisions, but APIs, GPU support, and parts of the integration remain experimental.

#kubernetes #ai agents #cncf #gpu scheduling

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
NVIDIA Developer Bloghttps://developer.nvidia.com/blog/?p=124728https://developer.nvidia.com/blog/agent-substrate-advancing-kubernetes-for-agents-at-scale/ Read discussion