Running AV Reasoning Models Within Onboard Compute and Latency Limits
Autonomous vehicle (AV) developers need reasoning models that can operate within the compute and latency constraints of an onboard system. Larger models provide...
First seen by Oriel on .
AI summary
NVIDIA presents Alpamayo 2 Edge, a compact reasoning vision-language-action model optimized for real-time autonomous driving on DRIVE AGX Thor, alongside AlpaJudge workflows for evaluating and improving driving rationales.
- Alpamayo 2 Edge combines a 2B Nemotron-based reasoning backbone with a diffusion action expert, using camera observations, temporal context, and navigation inputs to generate causal reasoning and trajectories.
- Adaptive reasoning can emit <NOMINAL> for routine driving instead of generating a reasoning trace; closed-loop reinforcement learning improved driving performance by 48.9% over the best checkpoint without it.
- The deployment recipe uses DFlash2, FP8 calibration, ONNX export, and TensorRT engines; measured pixel-to-trajectory latency is 160–190 ms depending on reasoning.
- AlpaJudge-RB scores alignment with a reference, while AlpaJudge-RF scores rationales using synchronized visual evidence and meta-action context; both require a CUDA GPU, and model weights need roughly 5 GB before activations.
- Judge scores can guide training-data selection and reinforcement-learning rewards, but low-scoring examples should be reviewed because they may represent rare failures worth learning from.
#autonomous vehicles #vision-language-action #reinforcement learning #tensorrt
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| NVIDIA Developer Blog | https://developer.nvidia.com/blog/?p=124882 | https://developer.nvidia.com/blog/running-av-reasoning-models-within-onboard-compute-and-latency-limits/ Read discussion |