English
Related papers

Related papers: GSR: Learning Structured Reasoning for Embodied Ma…

200 papers

Generative Recommendation (GR) has emerged as a transformative paradigm with its end-to-end generation advantages. However, existing GR methods primarily focus on direct Semantic ID (SID) generation from interaction sequences, failing to…

Information Retrieval · Computer Science 2026-05-19 Zihao Guo , Jian Wang , Ruxin Zhou , Youhua Liu , Jiawei Guo , Jun Zhao , Xiaoxiao Xu , Yongqi Liu , Kaiqiao Zhan

Mobile manipulators in households must both navigate and manipulate. This requires a compact, semantically rich scene representation that captures where objects are, how they function, and which parts are actionable. Scene graphs are a…

Computer Vision and Pattern Recognition · Computer Science 2026-02-10 Yuanchen Ju , Yongyuan Liang , Yen-Jen Wang , Nandiraju Gireesh , Yuanliang Ju , Seungjae Lee , Qiao Gu , Elvis Hsieh , Furong Huang , Koushil Sreenath

Standard game theory explains cooperation in repeated games through conditional strategies such as Tit-for-Tat (TfT), but these require continuous computation that imposes physical costs on embodied agents. We propose a three-layer…

Computer Science and Game Theory · Computer Science 2026-02-25 Yuki Nakamura

Embodied scene understanding requires not only comprehending visual-spatial information that has been observed but also determining where to explore next in the 3D physical world. Existing 3D Vision-Language (3D-VL) models primarily focus…

Computer Vision and Pattern Recognition · Computer Science 2025-07-31 Ziyu Zhu , Xilin Wang , Yixuan Li , Zhuofan Zhang , Xiaojian Ma , Yixin Chen , Baoxiong Jia , Wei Liang , Qian Yu , Zhidong Deng , Siyuan Huang , Qing Li

Software Engineering (SE) agents have shown promising abilities in supporting various SE tasks. Current SE agents remain fundamentally reactive, making decisions mainly based on conversation history and the most recent response. However,…

Software Engineering · Computer Science 2026-02-05 Tse-Hsun , Chen

Grounded video description (GVD) encourages captioning models to attend to appropriate video regions (e.g., objects) dynamically and generate a description. Such a setting can help explain the decisions of captioning models and prevents the…

Computer Vision and Pattern Recognition · Computer Science 2021-12-03 Wenqiao Zhang , Xin Eric Wang , Siliang Tang , Haizhou Shi , Haocheng Shi , Jun Xiao , Yueting Zhuang , William Yang Wang

Methods that use Large Language Models (LLM) as planners for embodied instruction following tasks have become widespread. To successfully complete tasks, the LLM must be grounded in the environment in which the robot operates. One solution…

Robotics · Computer Science 2025-12-25 Anatoly O. Onishchenko , Alexey K. Kovalev , Aleksandr I. Panov

Deploying autonomous robots that can learn new skills from demonstrations is an important challenge of modern robotics. Existing solutions often apply end-to-end imitation learning with Vision-Language Action (VLA) models or symbolic…

Robotics · Computer Science 2025-11-07 Maëlic Neau , Zoe Falomir , Paulo E. Santos , Anne-Gwenn Bosser , Cédric Buche

While Multimodal Large Language Models (MLLMs) have advanced GUI navigation agents, current approaches face limitations in cross-domain generalization and effective history utilization. We present a reasoning-enhanced framework that…

Artificial Intelligence · Computer Science 2025-11-03 Tao Liu , Chongyu Wang , Rongjie Li , Yingchen Yu , Xuming He , Bai Song

A key challenge in artificial intelligence and neuroscience is understanding how neural systems learn representations that capture the underlying dynamics of the world. Most world models represent the transition function with unstructured…

Machine Learning · Computer Science 2026-02-26 William Youngwoo Chung , Calvin Yeung , Hansen Jin Lillemark , Zhuowen Zou , Xiangjian Liu , Mohsen Imani

Scaling unsupervised skill discovery algorithms to high-DoF agents remains challenging. As dimensionality increases, the exploration space grows exponentially, while the manifold of meaningful skills remains limited. Therefore, semantic…

Machine Learning · Computer Science 2026-03-03 Seungeun Rho , Aaron Trinh , Danfei Xu , Sehoon Ha

Embodied Artificial Intelligence (Embodied AI) is crucial for achieving Artificial General Intelligence (AGI) and serves as a foundation for various applications (e.g., intelligent mechatronics systems, smart manufacturing) that bridge…

Computer Vision and Pattern Recognition · Computer Science 2025-08-26 Yang Liu , Weixing Chen , Yongjie Bai , Xiaodan Liang , Guanbin Li , Wen Gao , Liang Lin

Large language models (LLMs) excel at complex reasoning but remain limited by static and incomplete parametric knowledge. Retrieval-augmented generation (RAG) mitigates this by incorporating external knowledge, yet existing RAGs struggle…

Perspective-Aware AI requires modeling evolving internal states--goals, emotions, contexts--not merely preferences. Progress is limited by a data bottleneck: digital footprints are privacy-sensitive and perspective states are rarely…

Artificial Intelligence · Computer Science 2026-02-17 Jisung Shin , Daniel Platnick , Marjan Alirezaie , Hossein Rahnama

In complex embodied long-horizon manipulation tasks, effective task decomposition and execution require synergistic integration of textual logical reasoning and visual-spatial imagination to ensure efficient and accurate operation. Current…

Computer Vision and Pattern Recognition · Computer Science 2025-11-11 Xinyan Cai , Shiguang Wu , Dafeng Chi , Yuzheng Zhuang , Xingyue Quan , Jianye Hao , Qiang Guan

Vision-Language Models have excelled at textual reasoning, but they often struggle with fine-grained spatial understanding and continuous action planning, failing to simulate the dynamics required for complex visual reasoning. In this work,…

Foundation model-driven agents often struggle with long-horizon planning due to the transient nature of purely prompting-based reasoning. While existing skill induction methods mitigate this by distilling experience into state-blind…

Artificial Intelligence · Computer Science 2026-05-05 Jie-Jing Shao , Haiyan Yin , Yueming Lyu , Xingrui Yu , Lan-Zhe Guo , Ivor Tsang , James Kwok , Yu-Feng Li

Recent Graphical User Interface (GUI) agents replicate the R1-Zero paradigm, coupling online Reinforcement Learning (RL) with explicit chain-of-thought reasoning prior to object grounding and thereby achieving substantial performance gains.…

Computation and Language · Computer Science 2025-05-23 Yuqi Zhou , Sunhao Dai , Shuai Wang , Kaiwen Zhou , Qinglin Jia , Jun Xu

Achieving generalizable and precise robotic manipulation across diverse environments remains a critical challenge, largely due to limitations in spatial perception. While prior imitation-learning approaches have made progress, their…

Robotics · Computer Science 2025-05-28 Yiqi Huang , Travis Davies , Jiahuan Yan , Jiankai Sun , Xiang Chen , Luhui Hu

There is no limit to how much a robot might explore and learn, but all of that knowledge needs to be searchable and actionable. Within language research, retrieval augmented generation (RAG) has become the workhorse of large-scale…