Oriel — AI & technology worth noticing, every day.

Distributed training with Ray Train DLCs and EFA on EKS

Web page · https://aws.amazon.com/blogs/machine-learning/distributed-training-with-ray-train-dlcs-and-efa-on-eks

The Ray Train Deep Learning Container ships the full CUDA-through-EFA stack, so you build no image. This post walks through a reproducible sample that fine-tunes Qwen2.5-1.5B with PyTorch FSDP across eight GPUs on two EFA-enabled g6.12xlarge instances on Amazon EKS, using the container exactly as published.

First seen by Oriel on .

AI summary

AWS demonstrates running Qwen2.5-1.5B FSDP training across eight NVIDIA L4 GPUs on EKS using Ray Train DLCs and EFA, without building a custom container.

  • The public train-ml-cuda-v1.1 image bundles Ray 2.58.0, PyTorch 2.13.0, CUDA 13.0.2, NCCL 2.29.7, EFA libraries, aws-ofi-nccl, and diagnostics.
  • KubeRay creates a CPU-hosted Ray head and two four-GPU worker pods on g6.12xlarge instances within one Availability Zone.
  • FSDP shards approximately 12 GB of weights, gradients, and optimizer state eight ways, leaving roughly 1.5 GB per GPU while increasing inter-rank communication.
  • Pods explicitly request four GPUs and one EFA interface, while FI_PROVIDER and NCCL_NET_PLUGIN select EFA; missing EFA access causes NCCL to fall back to sockets.
  • The setup requires 96 G-family On-Demand vCPUs and costs about $9.50 hourly, including GPU instances, the EKS control plane, and NAT Gateway.

#ray train #amazon eks #efa #pytorch fsdp

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
AWS Machine Learning Blogad7c54a5a92c3c3ab550b696c83e5204108e2c13https://aws.amazon.com/blogs/machine-learning/distributed-training-with-ray-train-dlcs-and-efa-on-eks/ Read discussion