Make this forward pass as fast as possible
Web page · https://ianbarber.blog/2026/11/09/make-this-forward-pass-as-fast-as-possible
First seen by Oriel on .
AI summary
This essay argues that AI agents are not eliminating ML compilers, but decomposing them into infrastructure for faster optimization search, empirical measurement, shared abstractions, and correctness verification.
- TVM proposes a minimal, stable compiler foundation with shared IR, memory and synchronization checkers, and Lean formalization to make agent-generated kernels cheaper to verify.
- Together Compiler (TKCC) found unrestricted source access raised the performance ceiling, while agents spent about 20% of their time understanding engine codepaths and kernel calls.
- TKCC combines analytical top-k parallelism ranking, tiered hardware measurements, normalized forward-pass execution, and hot-swapping to narrow costly agentic optimization searches.
- OpenAI used Codex to remove Triton heuristics such as automatic FMA, making optimization behavior more explicit and the search landscape smoother, though potentially larger.
- Together reported knowledge hubs reduced speedup 25–31%, expert-written strategies lost to a simple optimization prompt, and tooling consumed up to 43% of agents’ calls.
#ml compilers #kernel optimization #ai agents #triton
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Points
Score reported by the source.
- Hacker News: 31, latest observation ; +28 over observed period since
Comments
Comment count reported by the source.
- Hacker News: 6, latest observation ; +6 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hacker News | 50435962 | https://ianbarber.blog/2026/11/09/make-this-forward-pass-as-fast-as-possible/ Read discussion |