English
Related papers

Related papers: ESI-Bench: Towards Embodied Spatial Intelligence t…

200 papers

In the realm of computer vision and robotics, embodied agents are expected to explore their environment and carry out human instructions. This necessitates the ability to fully understand 3D scenes given their first-person observations and…

Computer Vision and Pattern Recognition · Computer Science 2023-12-27 Tai Wang , Xiaohan Mao , Chenming Zhu , Runsen Xu , Ruiyuan Lyu , Peisen Li , Xiao Chen , Wenwei Zhang , Kai Chen , Tianfan Xue , Xihui Liu , Cewu Lu , Dahua Lin , Jiangmiao Pang

In this paper, we explore how we can build upon the data and models of Internet images and use them to adapt to robot vision without requiring any extra labels. We present a framework called Self-supervised Embodied Active Learning (SEAL).…

Computer Vision and Pattern Recognition · Computer Science 2021-12-03 Devendra Singh Chaplot , Murtaza Dalal , Saurabh Gupta , Jitendra Malik , Ruslan Salakhutdinov

Leveraging Multi-modal Large Language Models (MLLMs) to create embodied agents offers a promising avenue for tackling real-world tasks. While language-centric embodied agents have garnered substantial attention, MLLM-based embodied agents…

While large vision-language models (VLMs) are increasingly adopted as the perceptual backbone for embodied agents, existing benchmarks often rely on question-answering or multiple-choice formats. These protocols allow models to exploit…

Computer Vision and Pattern Recognition · Computer Science 2026-05-19 Haozhe Shan , Xiancong Ren , Han Dong , Haoyuan Shi , Yingji Zhang , Jiayu Hu , Yi Zhang , Yong Dai , Bin Shen , Lizhen Qu , Zenglin Xu , Xiaozhu Ju

Multimodal large language models (MLLMs) have advanced static visual--spatial reasoning, yet they often fail to preserve long-horizon spatial coherence in embodied settings where beliefs must be continuously revised from egocentric…

Computer Vision and Pattern Recognition · Computer Science 2026-05-29 Chih-Ting Liao , Xi Xiao , Chunlei Meng , Zhangquan Chen , Yitong Qiao , Weilin Zhou , Tianyang Wang , Xu Zheng , Xin Cao

Spatial cognition enables adaptive goal-directed behavior by constructing internal models of space. Robust biological systems consolidate spatial knowledge into three interconnected forms: \textit{landmarks} for salient cues, \textit{route…

Artificial Intelligence · Computer Science 2025-08-26 Shouwei Ruan , Liyuan Wang , Caixin Kang , Qihui Zhu , Songming Liu , Xingxing Wei , Hang Su

Embodied agents are expected to assist humans by actively exploring unknown environments and reasoning about spatial contexts. When deployed in real life, agents often face sequential tasks where each new task follows the completion of the…

Computer Vision and Pattern Recognition · Computer Science 2026-03-19 Zhongyi Cai , Yi Du , Chen Wang , Yu Kong

Spatial cognition is fundamental to real-world multimodal intelligence, allowing models to effectively interact with the physical environment. While multimodal large language models (MLLMs) have made significant strides, existing benchmarks…

Artificial Intelligence · Computer Science 2026-05-08 Peiran Xu , Sudong Wang , Yao Zhu , Jianing Li , Gege Qi , Yunjian Zhang

Artificial intelligence has demonstrated remarkable capability in predicting scientific properties, yet scientific discovery remains an inherently physical, long-horizon pursuit governed by experimental cycles. Most current computational…

Artificial Intelligence · Computer Science 2026-03-23 Xiang Zhuang , Chenyi Zhou , Kehua Feng , Zhihui Zhu , Yunfan Gao , Yijie Zhong , Yichi Zhang , Junjie Huang , Keyan Ding , Lei Bai , Haofen Wang , Qiang Zhang , Huajun Chen

Humans build viewpoint-independent cognitive maps through navigation, enabling intuitive reasoning about object permanence and spatial relations. We argue that multimodal large language models (MLLMs), despite extensive video training, lack…

Machine Learning · Computer Science 2025-12-02 Jacob Thompson , Emiliano Garcia-Lopez , Yonatan Bisk

The field of Embodied AI is witnessing a rapid evolution toward general-purpose robotic systems, fueled by high-fidelity simulation and large-scale data collection. However, this scaling capability remains severely bottlenecked by a…

Artificial Intelligence · Computer Science 2026-01-30 Zixing Lei , Genjia Liu , Yuanshuo Zhang , Qipeng Liu , Chuan Wen , Shanghang Zhang , Wenzhao Lian , Siheng Chen

Embodied intelligence is advancing rapidly, driving the need for efficient evaluation. Current benchmarks typically rely on interactive simulated environments or real-world setups, which are costly, fragmented, and hard to scale. To address…

Computer Vision and Pattern Recognition · Computer Science 2025-08-12 Jiahao Xiao , Jianbo Zhang , BoWen Yan , Shengyu Guo , Tongrui Ye , Kaiwei Zhang , Zicheng Zhang , Xiaohong Liu , Zhengxue Cheng , Lei Fan , Chuyi Li , Guangtao Zhai

Embodied artificial intelligence emphasizes the role of an agent's body in generating human-like behaviors. The recent efforts on EmbodiedAI pay a lot of attention to building up machine learning models to possess perceiving, planning, and…

Artificial Intelligence · Computer Science 2024-10-15 Chen Gao , Baining Zhao , Weichen Zhang , Jinzhu Mao , Jun Zhang , Zhiheng Zheng , Fanhang Man , Jianjie Fang , Zile Zhou , Jinqiang Cui , Xinlei Chen , Yong Li

Active event perception, the ability to dynamically detect, track, and summarize events in real time, is essential for embodied intelligence in tasks such as human-AI collaboration, assistive robotics, and autonomous navigation. However,…

Robotics · Computer Science 2025-06-24 Zhou Chen , Sanjoy Kundu , Harsimran S. Baweja , Sathyanarayanan N. Aakur

Spatial intelligence is essential for multimodal large language models, yet current benchmarks largely assess it only from an understanding perspective. We ask whether modern generative or unified multimodal models also possess generative…

Computer Vision and Pattern Recognition · Computer Science 2026-04-23 Muzhi Zhu , Shunyao Jiang , Huanyi Zheng , Zekai Luo , Hao Zhong , Anzhou Li , Kaijun Wang , Jintao Rong , Yang Liu , Hao Chen , Tao Lin , Chunhua Shen

Over the past year, the development of large language models (LLMs) has brought spatial intelligence into focus, with much attention on vision-based embodied intelligence. However, spatial intelligence spans a broader range of disciplines…

As agentic AI systems increasingly operate autonomously, establishing trust through verifiable evaluation becomes critical. Yet existing benchmarks lack the transparency and auditability needed to assess whether agents behave reliably. We…

Computation and Language · Computer Science 2025-12-02 Hyunjun Kim , Sooyoung Ryu

Spatial intelligence is central to embodied cognition, yet contemporary AI systems still struggle to reason about physical interactions in open-world human environments. Despite strong performance on controlled benchmarks, vision-language…

Computer Vision and Pattern Recognition · Computer Science 2026-03-24 Tianjun Gu , Jingyu Gong , Zhizhong Zhang , Yuan Xie , Lizhuang Ma , Xin Tan , Athanasios V

Despite the ubiquity of large language models (LLMs) in AI research, the question of embodiment in LLMs remains underexplored, distinguishing them from embodied systems in robotics where sensory perception directly informs physical action.…

Computation and Language · Computer Science 2024-05-28 Philipp Wicke , Lennart Wachowiak

The realization of Artificial General Intelligence (AGI) necessitates Embodied AI agents capable of robust spatial perception, effective task planning, and adaptive execution in physical environments. However, current large language models…