Oriel — AI & technology worth noticing, every day.

Train Qwen3 on AWS Trainium with torchtitan

Web page · https://aws.amazon.com/blogs/machine-learning/train-qwen3-on-aws-trainium-with-torchtitan

Train Qwen3 on AWS Trainium with torchtitan three ways on a single trn2.48xlarge: the Hugging Face Transformers v5 modeling backend, torchtitan's native Qwen3, and NKI mixture-of-experts kernels. The same torchrun launcher and config structure work unchanged across all three paths, including loading and saving published Hugging Face safetensors.

First seen by Oriel on .

AI summary

The post demonstrates three ways to train Qwen3 on AWS Trainium with torchtitan: Transformers v5, torchtitan’s native model, and NKI mixture-of-experts kernels.

  • All examples use a single trn2.48xlarge with 64 logical NeuronCore processes and a shared torchrun launch pattern.
  • The Transformers v5 path loads Hub architecture and safetensors through AutoModelForCausalLM; a verified fine-tuning run began at loss 6.54, versus 12.43 from random initialization.
  • The native path selects torchtitan’s Qwen3 implementation with model_registry("4B", attn_backend="sdpa").
  • The MoE example uses NKI grouped-experts kernels with 64 experts and top_k=4, but reduces Qwen3-30B-A3B to four layers to fit on one instance.
  • A custom NKI rotary-embedding kernel is registered as a PyTorch operator with autograd and a torchtitan override; the example demonstrates integration, not a performance improvement.

#aws trainium #torchtitan #qwen3 #nki

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
AWS Machine Learning Blog54e8299bd0c441c1532fc17ed8f18dacc33ccc0dhttps://aws.amazon.com/blogs/machine-learning/train-qwen3-on-aws-trainium-with-torchtitan/ Read discussion