Oriel — AI & technology worth noticing, every day.

Run Native DLA Inference in Python on NVIDIA Jetson AGX Orin Using cuDLA

Web page · https://developer.nvidia.com/blog/run-native-dla-inference-in-python-on-nvidia-jetson-agx-orin-using-cudla

AI applications at the edge often need to run multiple neural networks within limited compute and power budgets. NVIDIA Deep Learning Accelerator (DLA)...

First seen by Oriel on .

AI summary

This NVIDIA post demonstrates native DLA inference in Python on Jetson AGX Orin using cuda.bindings.cudla, combining GPU preprocessing, DLA execution, and GPU postprocessing through one CUDA stream.

  • cuda.bindings.cudla, introduced in cuda-bindings 13.3.0, exposes cuDLA runtime management directly to Python without requiring C++ orchestration.
  • JetPack supplies libcudla.so, while applications require Python 3.10 or newer and install the bindings with pip install "cuda-bindings>=13.3.0".
  • TensorRT trtexec compiles supported networks into standalone .bin or .nvdla loadables using flags such as --useDLACore=0 and --buildDLAStandalone.
  • The workflow introspects tensor descriptors, allocates and registers GPU buffers, submits a cudla.Task, synchronizes, and returns output bytes as a NumPy array.
  • Only hybrid mode is supported; DLA-compatible layers may require ONNX graph preprocessing, and every device, module, memory, and stream handle needs explicit cleanup.

#cudla #cuda python #jetson orin #deep learning inference

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
NVIDIA Developer Bloghttps://developer.nvidia.com/blog/?p=121180https://developer.nvidia.com/blog/run-native-dla-inference-in-python-on-nvidia-jetson-agx-orin-using-cudla/ Read discussion