ScaleNav: Scalable Internet Video Pretraining for Language-Conditioned Local Navigation
Paper · https://arxiv.org/abs/2610.25690
We study language-conditioned local navigation, where a robot must navigate to a text-specified target visible in its initial observation and stop at the described location. End-to-end navigation policies require diverse training data to generalize across scenes, yet real-robot data is costly to collect and simulated data leaves a sim-to-real gap. Internet egocentric videos offer a scalable source of real-world navigation experience. Our key insight is to learn goal-directed visual dynamics from Internet egocentric videos, and use these dynamics to support robot action generation. To this end, we propose ScaleNav, a unified world-action framework with three-stage training. We first perform instruction-conditioned video pretraining on these videos to learn how observations evolve as the camera approaches instructed targets across diverse real-world scenes. Action alignment then uses action-labeled simulated data to train the model to predict executable action chunks. Joint fine-tuning subsequently optimizes all trainable components together for navigation. ScaleNav achieves success rates of 86.5% on a single-view adaptation of Short-Horizon OVON and 94.9% on our HabitatGS-LN benchmark, outperforming the evaluated baselines on both benchmarks. We further deploy and evaluate ScaleNav on a real quadruped robot, with results supporting the model's ability to generalize to real-world navigation. Code and videos are available at https://zju3dv.github.io/ScaleNav/
First seen by Oriel on .
AI summary
ScaleNav is a language-conditioned robot navigation framework that learns visual dynamics from Internet egocentric videos, aligns them with executable actions, and outperforms evaluated baselines on two navigation benchmarks.
- The task requires a robot to approach a text-specified target visible in its initial observation and stop at the described location.
- Instruction-conditioned video pretraining learns how observations evolve when approaching targets; action alignment uses action-labeled simulated data, followed by joint fine-tuning of all trainable components.
- ScaleNav achieves success rates of 86.5% on a single-view adaptation of Short-Horizon OVON and 94.9% on HabitatGS-LN.
- Evaluation on a real quadruped robot supports real-world generalization, although the abstract provides no numerical deployment results.
#robot navigation #video pretraining #world models #sim-to-real
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Points
Score reported by the source.
- Hugging Face Papers: 86, latest observation ; +3 over observed period since
Comments
Comment count reported by the source.
- Hugging Face Papers: 2, latest observation ; 0 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Papers | 2610.25690 | https://arxiv.org/abs/2610.25690 Read discussion |