Memory-Enforced GPU Sharing in Kubernetes with NVIDIA CUDA 13.4
Web page · https://developer.nvidia.com/blog/memory-enforced-gpu-sharing-in-kubernetes-with-nvidia-cuda-13-4
Many AI workloads need only a fraction of a GPU. Giving each one an entire device can leave capacity idle; sharing GPUs lets platform teams serve more workloads...
First seen by Oriel on .
AI summary
CUDA 13.4 adds transparent GPU memory enforcement for shared workloads, while the open-source gpu-fractioning operator applies per-container memory and compute settings on Kubernetes nodes.
- Workloads can request guaranteed GPU memory with an optional higher limit, and choose time-slicing or sm-sharing compute modes.
- GPU Fractioning Operator runs MPS, configures workload limits from Pod annotations, and reports memory and compute use per fraction.
- KAI Scheduler v0.18.0 integrates the operator through its NvFractions mode; other schedulers can use the feature by writing the same annotations.
- Time-slicing runs workloads in turns with separate error domains; sm-sharing runs them concurrently but places them in a shared error domain.
- Memory requests can be specified as absolute sizes or GPU portions, and can apply independently to multiple GPUs.
#kubernetes #gpu sharing #cuda #kai scheduler
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| NVIDIA Developer Blog | https://developer.nvidia.com/blog/?p=124247 | https://developer.nvidia.com/blog/memory-enforced-gpu-sharing-in-kubernetes-with-nvidia-cuda-13-4/ Read discussion |