English
Related papers

Related papers: Embodied4C: Measuring What Matters for Embodied Vi…

200 papers

Recent advances in large vision-language models have led to impressive performance in visual question answering and multimodal reasoning. However, it remains unclear whether these models genuinely perform grounded visual reasoning or rely…

Computer Vision and Pattern Recognition · Computer Science 2025-07-11 Chengfei Wu , Ronald Seoh , Bingxuan Li , Liqiang Zhang , Fengrong Han , Dan Goldwasser

Vision-Language Navigation (VLN) is a core challenge in embodied AI, requiring agents to navigate real-world environments using natural language instructions. Current language model-based navigation systems operate on discrete topological…

Computer Vision and Pattern Recognition · Computer Science 2025-06-26 Zhangyang Qi , Zhixiong Zhang , Yizhou Yu , Jiaqi Wang , Hengshuang Zhao

Vision-Language Models (VLMs) have recently emerged as powerful tools, excelling in tasks that integrate visual and textual comprehension, such as image captioning, visual question answering, and image-text retrieval. However, existing…

Computer Vision and Pattern Recognition · Computer Science 2025-03-26 Ilias Stogiannidis , Steven McDonagh , Sotirios A. Tsaftaris

As reasoning models scale rapidly, the essential role of multimodality in human cognition has come into sharp relief, driving a growing need to probe vision-centric cognitive behaviors. Yet, existing multimodal benchmarks either…

General-purpose robotic systems operating in open-world environments must achieve both broad generalization and high-precision action execution, a combination that remains challenging for existing Vision-Language-Action (VLA) models. While…

Visual grounding, localizing objects from natural language descriptions, represents a critical bridge between language and vision understanding. While multimodal large language models (MLLMs) achieve impressive scores on existing…

Computer Vision and Pattern Recognition · Computer Science 2026-03-24 Rang Li , Lei Li , Shuhuai Ren , Hao Tian , Shuhao Gu , Shicheng Li , Zihao Yue , Yudong Wang , Wenhan Ma , Zhe Yang , Jingyuan Ma , Zhifang Sui , Fuli Luo

"Embodied visual navigation" problem requires an agent to navigate in a 3D environment mainly rely on its first-person observation. This problem has attracted rising attention in recent years due to its wide application in autonomous…

Robotics · Computer Science 2021-10-12 Fengda Zhu , Yi Zhu , Vincent CS Lee , Xiaodan Liang , Xiaojun Chang

Embodied Vision-Language Models (VLMs) have demonstrated impressive performance and generalization in robotics, particularly within Vision-Language-Action frameworks. However, a significant gap remains between the high-level semantic focus…

Computer Vision and Pattern Recognition · Computer Science 2026-05-28 Ruowen Zhao , Bangguo Li , Zuyan Liu , Yinan Liang , Junliang Ye , Fangfu Liu , Diankun Wu , Zhengyi Wang , Xumin Yu , Yongming Rao , Han Hu , Jun Zhu

We aim to evaluate Large Language Models (LLMs) for embodied decision making. While a significant body of work has been leveraging LLMs for decision making in embodied environments, we still lack a systematic understanding of their…

Vision-Language Models (VLMs) have shown remarkable progress in Vision-Language Navigation (VLN), offering new possibilities for navigation decision-making that could benefit both robotic platforms and human users. However, real-world…

Computer Vision and Pattern Recognition · Computer Science 2026-02-23 Xia Su , Ruiqi Chen , Benlin Liu , Jingwei Ma , Zonglin Di , Ranjay Krishna , Jon Froehlich

Traditional reinforcement learning-based robotic control methods are often task-specific and fail to generalize across diverse environments or unseen objects and instructions. Visual Language Models (VLMs) demonstrate strong scene…

Robotics · Computer Science 2024-12-18 Qi Sun , Pengfei Hong , Tej Deep Pala , Vernon Toh , U-Xuan Tan , Deepanway Ghosal , Soujanya Poria

Recent advances in multimodal large language models (MLLMs) have opened new opportunities for embodied intelligence, enabling multimodal understanding, reasoning, and interaction, as well as continuous spatial decision-making. Nevertheless,…

Multimodal Large Language Models (MLLMs) have shown significant advancements, providing a promising future for embodied agents. Existing benchmarks for evaluating MLLMs primarily utilize static images or videos, limiting assessments to…

Computer Vision and Pattern Recognition · Computer Science 2025-04-14 Zhili Cheng , Yuge Tu , Ran Li , Shiqi Dai , Jinyi Hu , Shengding Hu , Jiahao Li , Yang Shi , Tianyu Yu , Weize Chen , Lei Shi , Maosong Sun

While Vision-Language Models (VLMs) have advanced highlevel reasoning in autonomous driving, their ability to ground this reasoning in the underlying physics of ego-motion remains poorly understood. We introduce EgoDyn-Bench, a diagnostic…

Computer Vision and Pattern Recognition · Computer Science 2026-04-28 Finn Rasmus Schäfer , Yuan Gao , Dingrui Wang , Thomas Stauner , Stephan Günnemann , Mattia Piccinini , Sebastian Schmidt , Johannes Betz

Spatial reasoning is a fundamental capability for embodied intelligence, especially for fine-grained manipulation tasks such as robotic assembly. While recent vision-language models (VLMs) exhibit preliminary spatial awareness, they largely…

Robotics · Computer Science 2026-04-13 Zhi Jing , Jinbin Qiao , Ouyang Lu , Jicong Ao , Shuang Qiu , Yu-Gang Jiang , Chenjia Bai

Large vision-language models have recently demonstrated impressive performance in planning and control tasks, driving interest in their application to real-world robotics. However, deploying these models for reasoning in embodied contexts…

Computer Vision and Pattern Recognition · Computer Science 2025-09-30 Karmesh Yadav , Yusuf Ali , Gunshi Gupta , Yarin Gal , Zsolt Kira

Recent advancements in Multi-modal Large Language Models (MLLMs) have opened new avenues for applications in Embodied AI. Building on previous work, EgoThink, we introduce VidEgoThink, a comprehensive benchmark for evaluating egocentric…

Computer Vision and Pattern Recognition · Computer Science 2024-10-16 Sijie Cheng , Kechen Fang , Yangyang Yu , Sicheng Zhou , Bohao Li , Ye Tian , Tingguang Li , Lei Han , Yang Liu

Despite the remarkable progress of Vision-Language Models (VLMs) in adopting "Thinking-with-Images" capabilities, accurately evaluating the authenticity of their reasoning process remains a critical challenge. Existing benchmarks mainly…

Computer Vision and Pattern Recognition · Computer Science 2026-01-21 Xuchen Li , Xuzhao Li , Renjie Pi , Shiyu Hu , Jian Zhao , Jiahui Gao

Recent advances in deep thinking models have demonstrated remarkable reasoning capabilities on mathematical and coding tasks. However, their effectiveness in embodied domains which require continuous interaction with environments through…

Computation and Language · Computer Science 2025-05-15 Wenqi Zhang , Mengna Wang , Gangao Liu , Xu Huixin , Yiwei Jiang , Yongliang Shen , Guiyang Hou , Zhe Zheng , Hang Zhang , Xin Li , Weiming Lu , Peng Li , Yueting Zhuang

Existing evaluation frameworks for Multimodal Large Language Models (MLLMs) primarily focus on image reasoning or general video understanding tasks, largely overlooking the significant role of image context in video comprehension. To bridge…