English
Related papers

Related papers: Navigating Beyond Instructions: Vision-and-Languag…

200 papers

Vision-and-language navigation (VLN) agents are trained to navigate in real-world environments by following natural language instructions. A major challenge in VLN is the limited availability of training data, which hinders the models'…

Computer Vision and Pattern Recognition · Computer Science 2023-05-24 Zi-Yi Dou , Feng Gao , Nanyun Peng

The aspiration of the Vision-and-Language Navigation (VLN) task has long been to develop an embodied agent with robust adaptability, capable of seamlessly transferring its navigation capabilities across various tasks. Despite remarkable…

Computer Vision and Pattern Recognition · Computer Science 2025-03-19 Siqi Zhang , Yanyuan Qiao , Qunbo Wang , Longteng Guo , Zhihua Wei , Jing Liu

Vision-and-language navigation requires an agent to navigate through a real 3D environment following natural language instructions. Despite significant advances, few previous works are able to fully utilize the strong correspondence between…

Computer Vision and Pattern Recognition · Computer Science 2020-10-06 Yicong Hong , Cristian Rodriguez-Opazo , Qi Wu , Stephen Gould

Vision-and-Language Navigation (VLN) is a task where agents must decide how to move through a 3D environment to reach a goal by grounding natural language instructions to the visual surroundings. One of the problems of the VLN task is data…

Computer Vision and Pattern Recognition · Computer Science 2025-03-18 Tsu-Jui Fu , Xin Eric Wang , Matthew Peterson , Scott Grafton , Miguel Eckstein , William Yang Wang

Autonomous navigation guided by natural language instructions in embodied environments remains a challenge for vision-language navigation (VLN) agents. Although recent advancements in learning diverse and fine-grained visual environmental…

Computer Vision and Pattern Recognition · Computer Science 2025-04-08 Xuesong Zhang , Jia Li , Yunbo Xu , Zhenzhen Hu , Richang Hong

Vision-and-Language Navigation (VLN) has gained increasing attention over recent years and many approaches have emerged to advance their development. The remarkable achievements of foundation models have shaped the challenges and proposed…

Computation and Language · Computer Science 2024-12-31 Yue Zhang , Ziqiao Ma , Jialu Li , Yanyuan Qiao , Zun Wang , Joyce Chai , Qi Wu , Mohit Bansal , Parisa Kordjamshidi

Visual Language Navigation (VLN) is a fundamental task within the field of Embodied AI, focusing on the ability of agents to navigate complex environments based on natural language instructions. Despite the progress made by existing…

Computer Vision and Pattern Recognition · Computer Science 2025-06-23 Ruoyu Wang , Tong Yu , Junda Wu , Yao Liu , Julian McAuley , Lina Yao

The vision-language navigation (VLN) task requires an agent to reach a target with the guidance of natural language instruction. Previous works learn to navigate step-by-step following an instruction. However, these works may fail to…

Computer Vision and Pattern Recognition · Computer Science 2021-12-10 Xiwen Liang , Fengda Zhu , Yi Zhu , Bingqian Lin , Bing Wang , Xiaodan Liang

The challenging task of Vision-and-Language Navigation (VLN) requires embodied agents to follow natural language instructions to reach a goal location or object (e.g. `walk down the hallway and turn left at the piano'). For agents to…

Computer Vision and Pattern Recognition · Computer Science 2023-12-04 Meera Hahn , Amit Raj , James M. Rehg

Vision-and-Language Navigation (VLN) is unique in that it requires turning relatively general natural-language instructions into robot agent actions, on the basis of the visible environment. This requires to extract value from two very…

Computation and Language · Computer Science 2020-07-30 Yuankai Qi , Zizheng Pan , Shengping Zhang , Anton van den Hengel , Qi Wu

Vision-and-language navigation (VLN) asks an agent to follow a given language instruction to navigate through a real 3D environment. Despite significant advances, conventional VLN agents are trained typically under disturbance-free…

Computer Vision and Pattern Recognition · Computer Science 2024-03-12 Bingqian Lin , Yanxin Long , Yi Zhu , Fengda Zhu , Xiaodan Liang , Qixiang Ye , Liang Lin

Vision-and-Language Navigation (VLN) requires grounding instructions, such as "turn right and stop at the door", to routes in a visual environment. The actual grounding can connect language to the environment through multiple modalities,…

Computation and Language · Computer Science 2019-06-11 Ronghang Hu , Daniel Fried , Anna Rohrbach , Dan Klein , Trevor Darrell , Kate Saenko

Vision-Language Navigation (VLN) is a critical task for developing embodied agents that can follow natural language instructions to navigate in complex real-world environments. Recent advances in VLN by large pretrained models have…

Visual Language Navigation is a task that challenges robots to navigate in realistic environments based on natural language instructions. While previous research has largely focused on static settings, real-world navigation must often…

Computer Vision and Pattern Recognition · Computer Science 2026-01-07 Dillon Loh , Tomasz Bednarz , Xinxing Xia , Frank Guan

Conventional Vision-and-Language Navigation (VLN) benchmarks assume instructions are feasible and the referenced target exists, leaving agents ill-equipped to handle false-premise goals. We introduce VLN-NF, a benchmark with false-premise…

Robotics · Computer Science 2026-04-21 Hung-Ting Su , Ting-Jun Wang , Jia-Fong Yeh , Min Sun , Winston H. Hsu

As a long-term vision in the field of artificial intelligence, the core goal of embodied intelligence is to improve the perception, understanding, and interaction capabilities of agents and the environment. Vision-language navigation (VLN),…

Robotics · Computer Science 2024-03-21 Peng Gao , Peng Wang , Feng Gao , Fei Wang , Ruyue Yuan

Recent advances in Iterative Vision-and-Language Navigation (IVLN) introduce a more meaningful and practical paradigm of VLN by maintaining the agent's memory across tours of scenes. Although the long-term memory aligns better with the…

Computer Vision and Pattern Recognition · Computer Science 2024-03-27 Ganlong Zhao , Guanbin Li , Weikai Chen , Yizhou Yu

Vision-and-Language Navigation (VLN) is a challenging task in the field of artificial intelligence. Although massive progress has been made in this task over the past few years attributed to breakthroughs in deep vision and language models,…

Computer Vision and Pattern Recognition · Computer Science 2021-11-30 Wenda Qin , Teruhisa Misu , Derry Wijaya

While natural language is commonly used to guide embodied agents, the inherent ambiguity and verbosity of language often hinder the effectiveness of language-guided navigation in complex environments. To this end, we propose Visual Prompt…

Computer Vision and Pattern Recognition · Computer Science 2025-11-25 Shuo Feng , Zihan Wang , Yuchen Li , Rui Kong , Hengyi Cai , Shuaiqiang Wang , Gim Hee Lee , Piji Li , Shuqiang Jiang

Vision-and-Language Navigation (VLN) presents a complex challenge in embodied AI, requiring agents to interpret natural language instructions and navigate through visually rich, unfamiliar environments. Recent advances in large…

Robotics · Computer Science 2025-06-13 Yicheng Duan , Kaiyu tang