Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 2
The Global Performance Tuner is a feature for automatic performance tuning released in NVIDIA TensorRT 11.1. It provides automatic, accuracy-aware and...
First seen by Oriel on .
AI summary
The post explains TensorRT 11.1’s Global Performance Tuner, which searches alternative compiler build routes and selects the fastest measured engine that meets an application-defined accuracy threshold.
- Build routes are specified by combinations of tunable knob values; TensorRT exposes C++ APIs to apply a route and inspect available knobs and values.
- The trtexec tuner builds each candidate, measures accuracy and end-to-end performance, excludes failed or inaccurate candidates, and selects the fastest eligible engine.
- Across ten evaluated workloads, tuning improved performance by 1.14x to 1.63x over the default build route.
- Fast search varies one knob at a time, mixed search adds promising combinations, and full search evaluates the complete Cartesian product.
- Tuning results depend on the model, shapes, GPU, software stack, and TensorRT version; representative validation data and re-tuning after environment changes are recommended.
#tensorrt #gpu performance #compiler optimization #model accuracy
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| NVIDIA Developer Blog | https://developer.nvidia.com/blog/?p=124139 | https://developer.nvidia.com/blog/scaling-nvidia-tensorrt-performance-optimization-beyond-a-single-build-route-with-the-global-performance-tuner-part-2/ Read discussion |