Profiling TorchNeuron (Part 1): System-level profiling on AWS Trainium.
Web page · https://huggingface.co/blog/amazon/profiling-torchneuron-part-1
First seen by Oriel on .
AI summary
The post demonstrates system-level profiling of RMSNorm on AWS Trainium, showing how torch.rms_norm and torch.compile reduce device time by fusing operations.
- It configures torch.profiler with ProfilerActivity.PrivateUse1, NeuronConfig, synchronization, and NeuronProfiler to capture framework and Neuron traces.
- The naive implementation takes 2,355 µs of Neuron time across six operator calls; three aten::mul calls account for the largest share.
- torch.rms_norm runs as one fused kernel in 403 µs, while torch.compile runs the original function as one dispatch in 376 µs.
- Neuron Explorer links framework dispatch, runtime scheduling, and hardware execution, helping verify hotspots and see gaps removed by fusion.
- The workflow is intended for identifying inference bottlenecks; Part 2 will examine device-level engine breakdowns and roofline analysis.
#aws trainium #pytorch #torchneuron #ml profiling
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Blog | https://huggingface.co/blog/amazon/profiling-torchneuron-part-1 | https://huggingface.co/blog/amazon/profiling-torchneuron-part-1 Read discussion |