Run Native DLA Inference in Python on NVIDIA Jetson AGX Orin Using cuDLA
AI applications at the edge often need to run multiple neural networks within limited compute and power budgets. NVIDIA Deep Learning Accelerator (DLA)...
First seen by Oriel on .
AI summary
This NVIDIA post demonstrates native DLA inference in Python on Jetson AGX Orin using cuda.bindings.cudla, combining GPU preprocessing, DLA execution, and GPU postprocessing through one CUDA stream.
- cuda.bindings.cudla, introduced in cuda-bindings 13.3.0, exposes cuDLA runtime management directly to Python without requiring C++ orchestration.
- JetPack supplies libcudla.so, while applications require Python 3.10 or newer and install the bindings with pip install "cuda-bindings>=13.3.0".
- TensorRT trtexec compiles supported networks into standalone .bin or .nvdla loadables using flags such as --useDLACore=0 and --buildDLAStandalone.
- The workflow introspects tensor descriptors, allocates and registers GPU buffers, submits a cudla.Task, synchronizes, and returns output bytes as a NumPy array.
- Only hybrid mode is supported; DLA-compatible layers may require ONNX graph preprocessing, and every device, module, memory, and stream handle needs explicit cleanup.
#cudla #cuda python #jetson orin #deep learning inference
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| NVIDIA Developer Blog | https://developer.nvidia.com/blog/?p=121180 | https://developer.nvidia.com/blog/run-native-dla-inference-in-python-on-nvidia-jetson-agx-orin-using-cudla/ Read discussion |