English
Related papers

Related papers: On the Limits of Evaluating Embodied Agent Model G…

200 papers

The recently proposed ALFRED challenge task aims for a virtual robotic agent to complete complex multi-step everyday tasks in a virtual home environment from high-level natural language directives, such as "put a hot piece of bread on a…

Computation and Language · Computer Science 2020-10-28 Peter A. Jansen

Recent advances in vision-language models (VLMs) have shown promise for human-level embodied intelligence. However, existing benchmarks for VLM-driven embodied agents often rely on high-level commands or discretized action spaces, which are…

Artificial Intelligence · Computer Science 2026-02-25 Bo Peng , Pi Bu , Keyu Pan , Xinrun Xu , Yinxiu Zhao , Miao Chen , Yang Du , Lin Li , Jun Song , Tong Xu

Language-guided Embodied AI benchmarks requiring an agent to navigate an environment and manipulate objects typically allow one-way communication: the human user gives a natural language command to the agent, and the agent can only follow…

Artificial Intelligence · Computer Science 2022-08-17 Xiaofeng Gao , Qiaozi Gao , Ran Gong , Kaixiang Lin , Govind Thattai , Gaurav S. Sukhatme

Recent research on instructable agents has used memory-augmented Large Language Models (LLMs) as task planners, a technique that retrieves language-program examples relevant to the input instruction and uses them as in-context examples in…

Artificial Intelligence · Computer Science 2024-05-01 Gabriel Sarch , Sahil Somani , Raghav Kapoor , Michael J. Tarr , Katerina Fragkiadaki

Generalization to unseen concepts is a central challenge due to the scarcity of human annotations in Mention-agnostic Biomedical Concept Recognition (MA-BCR). This work makes two key contributions to systematically address this issue.…

Embodied agents operating in the physical world must make decisions that are not only effective but also safe, spatially coherent, and grounded in context. While recent advances in large multimodal models (LMMs) have shown promising…

Computer Vision and Pattern Recognition · Computer Science 2025-09-23 Dinura Dissanayake , Ahmed Heakl , Omkar Thawakar , Noor Ahsan , Ritesh Thawkar , Ketan More , Jean Lahoud , Rao Anwer , Hisham Cholakkal , Ivan Laptev , Fahad Shahbaz Khan , Salman Khan

Benchmarks for language-guided embodied agents typically assume text-based instructions, but deployed agents will encounter spoken instructions. While Automatic Speech Recognition (ASR) models can bridge the input gap, erroneous ASR…

Computation and Language · Computer Science 2023-10-11 Allen Chang , Xiaoyuan Zhu , Aarav Monga , Seoho Ahn , Tejas Srinivasan , Jesse Thomason

An interactive instruction following task has been proposed as a benchmark for learning to map natural language instructions and first-person vision into sequences of actions to interact with objects in 3D environments. We found that an…

Artificial Intelligence · Computer Science 2022-11-16 Kazutoshi Shinoda , Yuki Takezawa , Masahiro Suzuki , Yusuke Iwasawa , Yutaka Matsuo

Current approaches to embodied AI tend to learn policies from expert demonstrations. However, without a mechanism to evaluate the quality of demonstrated actions, they are limited to learning from optimal behaviour, or they risk replicating…

Computation and Language · Computer Science 2025-10-14 Sabrina McCallum , Amit Parekh , Alessandro Suglia

We introduce a simple method that employs pre-trained CLIP encoders to enhance model generalization in the ALFRED task. In contrast to previous literature where CLIP replaces the visual encoder, we suggest using CLIP as an additional module…

Computer Vision and Pattern Recognition · Computer Science 2024-06-27 Ye Won Byun , Cathy Jiao , Shahriar Noroozizadeh , Jimin Sun , Rosa Vitiello

Vision-language models (VLMs) have shown strong perception and reasoning abilities for instruction-following embodied agents. However, despite these abilities and their generalization performance, they still face limitations in…

Computer Vision and Pattern Recognition · Computer Science 2026-04-23 Jinsik Bang , Jaeyeon Bae , Donggyu Lee , Siyeol Jung , Taehwan Kim

Embodied Instruction Following (EIF) studies how autonomous mobile manipulation robots should be controlled to accomplish long-horizon tasks described by natural language instructions. While much research on EIF is conducted in simulators,…

Robotics · Computer Science 2024-03-13 Yuki Inoue , Hiroki Ohashi

Recent progress in embodied AI has produced a growing ecosystem of robot policies, foundation models, and modular runtimes. However, current evaluation remains dominated by task success metrics such as completion rate or manipulation…

Robotics · Computer Science 2026-04-14 Xue Qin , Simin Luan , John See , Cong Yang , Zhijun Li

Language-conditioned robot models have the potential to enable robots to perform a wide range of tasks based on natural language instructions. However, assessing their safety and effectiveness remains challenging because it is difficult to…

End-to-end Transformers have demonstrated an impressive success rate for Embodied Instruction Following when the environment has been seen in training. However, they tend to struggle when deployed in an unseen environment. This lack of…

Computation and Language · Computer Science 2023-10-20 Cheng-Fu Yang , Yen-Chun Chen , Jianwei Yang , Xiyang Dai , Lu Yuan , Yu-Chiang Frank Wang , Kai-Wei Chang

People always desire an embodied agent that can perform a task by understanding language instruction. Moreover, they also want to monitor and expect agents to understand commands the way they expected. But, how to build such an embodied…

Artificial Intelligence · Computer Science 2022-03-10 Haoyu Liu , Yang Liu , Hongkai He , Hangfang Yang

Embodied agents operating in household environments must interpret ambiguous and under-specified human instructions. A capable household robot should recognize ambiguity and ask relevant clarification questions to infer the user intent…

Artificial Intelligence · Computer Science 2025-10-06 Ram Ramrakhya , Matthew Chang , Xavier Puig , Ruta Desai , Zsolt Kira , Roozbeh Mottaghi

Embodied AI Agents are quickly becoming important and common tools in society. These embodied agents should be able to learn about and accomplish a wide range of user goals and preferences efficiently and robustly. Large Language Models…

Artificial Intelligence · Computer Science 2026-02-20 Rachel Ma , Jingyi Qu , Andreea Bobu , Dylan Hadfield-Menell

Humans are excellent at understanding language and vision to accomplish a wide range of tasks. In contrast, creating general instruction-following embodied agents remains a difficult challenge. Prior work that uses pure language-only models…

Computer Vision and Pattern Recognition · Computer Science 2023-03-28 Hao Liu , Lisa Lee , Kimin Lee , Pieter Abbeel

While the exploration for embodied AI has spanned multiple decades, it remains a persistent challenge to endow agents with human-level intelligence, including perception, learning, reasoning, decision-making, control, and generalization…

Robotics · Computer Science 2024-02-07 Zhiyuan Xu , Kun Wu , Junjie Wen , Jinming Li , Ning Liu , Zhengping Che , Jian Tang