Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKS
NVIDIA Nemotron 3.5 Super VL is a 120B-parameter mixture-of-experts model that pairs reasoning and tool use with multimodal understanding. This post explores its architecture and use cases, then shows how to deploy it on Amazon EKS, EKS Auto Mode, and Amazon SageMaker HyperPod EKS with NVIDIA Dynamo and vLLM.
First seen by Oriel on .
AI summary
AWS describes NVIDIA Nemotron 3.5 Super VL, a 120B-parameter open multimodal MoE model, and deployment, validation, and capacity-planning patterns for agentic AI on Amazon EKS and SageMaker HyperPod EKS.
- Nemotron 3.5 Super VL activates 12B parameters per token, accepts text, images, documents, and video, and produces text using BF16 or NVFP4 checkpoints.
- Its hybrid Mamba-2, attention, and MoE architecture supports 1M-token contexts, while constant-size Mamba-2 state reduces generation memory growth.
- NVIDIA reports scores of 63.44 percent on Hermes Bench, 87.05 percent on LangChain agent evaluations, and 80.8 percent on CharXiv.
- Minimum vLLM configurations range from one B300/GB300 GPU with NVFP4 to 16 H100 GPUs with BF16.
- AWS deploys the model with NVIDIA Dynamo and vLLM across Amazon EKS options, using ECR, FSx for Lustre, EFA, and GPU nodes; aiperf measures TTFT, ITL, and throughput.
#nemotron #amazon eks #vllm #multimodal ai
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| AWS Machine Learning Blog | 3b5d85c9b7debe771c9397243d2a7ae65f84c3cf | https://aws.amazon.com/blogs/machine-learning/accelerate-agentic-ai-with-nvidia-nemotron-models-on-amazon-eks-and-sagemaker-hyperpod-eks/ Read discussion |