Oriel — AI & technology worth noticing, every day.

Profiling TorchNeuron (Part 1): System-level profiling on AWS Trainium.

Web page · https://huggingface.co/blog/amazon/profiling-torchneuron-part-1

First seen by Oriel on .

AI summary

The post demonstrates system-level profiling of RMSNorm on AWS Trainium, showing how torch.rms_norm and torch.compile reduce device time by fusing operations.

  • It configures torch.profiler with ProfilerActivity.PrivateUse1, NeuronConfig, synchronization, and NeuronProfiler to capture framework and Neuron traces.
  • The naive implementation takes 2,355 µs of Neuron time across six operator calls; three aten::mul calls account for the largest share.
  • torch.rms_norm runs as one fused kernel in 403 µs, while torch.compile runs the original function as one dispatch in 376 µs.
  • Neuron Explorer links framework dispatch, runtime scheduling, and hardware execution, helping verify hotspots and see gaps removed by fusion.
  • The workflow is intended for identifying inference bottlenecks; Part 2 will examine device-level engine breakdowns and roofline analysis.

#aws trainium #pytorch #torchneuron #ml profiling

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
Hugging Face Bloghttps://huggingface.co/blog/amazon/profiling-torchneuron-part-1https://huggingface.co/blog/amazon/profiling-torchneuron-part-1 Read discussion