Distributed training with Ray Train DLCs and EFA on EKS
The Ray Train Deep Learning Container ships the full CUDA-through-EFA stack, so you build no image. This post walks through a reproducible sample that fine-tunes Qwen2.5-1.5B with PyTorch FSDP across eight GPUs on two EFA-enabled g6.12xlarge instances on Amazon EKS, using the container exactly as published.
First seen by Oriel on .
AI summary
AWS demonstrates running Qwen2.5-1.5B FSDP training across eight NVIDIA L4 GPUs on EKS using Ray Train DLCs and EFA, without building a custom container.
- The public train-ml-cuda-v1.1 image bundles Ray 2.58.0, PyTorch 2.13.0, CUDA 13.0.2, NCCL 2.29.7, EFA libraries, aws-ofi-nccl, and diagnostics.
- KubeRay creates a CPU-hosted Ray head and two four-GPU worker pods on g6.12xlarge instances within one Availability Zone.
- FSDP shards approximately 12 GB of weights, gradients, and optimizer state eight ways, leaving roughly 1.5 GB per GPU while increasing inter-rank communication.
- Pods explicitly request four GPUs and one EFA interface, while FI_PROVIDER and NCCL_NET_PLUGIN select EFA; missing EFA access causes NCCL to fall back to sockets.
- The setup requires 96 G-family On-Demand vCPUs and costs about $9.50 hourly, including GPU instances, the EKS control plane, and NAT Gateway.
#ray train #amazon eks #efa #pytorch fsdp
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| AWS Machine Learning Blog | ad7c54a5a92c3c3ab550b696c83e5204108e2c13 | https://aws.amazon.com/blogs/machine-learning/distributed-training-with-ray-train-dlcs-and-efa-on-eks/ Read discussion |