English

DreamNav: A Trajectory-Based Imaginative Framework for Zero-Shot Vision-and-Language Navigation

Robotics 2025-09-16 v1 Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition

Abstract

Vision-and-Language Navigation in Continuous Environments (VLN-CE), which links language instructions to perception and control in the real world, is a core capability of embodied robots. Recently, large-scale pretrained foundation models have been leveraged as shared priors for perception, reasoning, and action, enabling zero-shot VLN without task-specific training. However, existing zero-shot VLN methods depend on costly perception and passive scene understanding, collapsing control to point-level choices. As a result, they are expensive to deploy, misaligned in action semantics, and short-sighted in planning. To address these issues, we present DreamNav that focuses on the following three aspects: (1) for reducing sensory cost, our EgoView Corrector aligns viewpoints and stabilizes egocentric perception; (2) instead of point-level actions, our Trajectory Predictor favors global trajectory-level planning to better align with instruction semantics; and (3) to enable anticipatory and long-horizon planning, we propose an Imagination Predictor to endow the agent with proactive thinking capability. On VLN-CE and real-world tests, DreamNav sets a new zero-shot state-of-the-art (SOTA), outperforming the strongest egocentric baseline with extra information by up to 7.49\% and 18.15\% in terms of SR and SPL metrics. To our knowledge, this is the first zero-shot VLN method to unify trajectory-level planning and active imagination while using only egocentric inputs.

Keywords

Cite

@article{arxiv.2509.11197,
  title  = {DreamNav: A Trajectory-Based Imaginative Framework for Zero-Shot Vision-and-Language Navigation},
  author = {Yunheng Wang and Yuetong Fang and Taowen Wang and Yixiao Feng and Yawen Tan and Shuning Zhang and Peiran Liu and Yiding Ji and Renjing Xu},
  journal= {arXiv preprint arXiv:2509.11197},
  year   = {2025}
}
R2 v1 2026-07-01T05:35:21.815Z