English
Related papers

Related papers: Building Open-Ended Embodied Agent via Language-Po…

200 papers

Large Language Models (LLMs) offer a promising basis for creating agents that can tackle complex tasks through iterative environmental interaction. Existing methods either require these agents to mimic expert-provided trajectories or rely…

Computation and Language · Computer Science 2024-12-02 Dihong Gong , Pu Lu , Zelong Wang , Meng Zhou , Xiuqiang He

Large language model (LLM) agents are moving beyond prompting alone. ChatGPT marked the rise of general-purpose LLM assistants, DeepSeek showed that on-policy reinforcement learning with verifiable rewards can improve reasoning and tool…

Large Language Models (LLMs) have demonstrated excellent capabilities in composing various modules together to create programs that can perform complex reasoning tasks on images. In this paper, we propose TANGO, an approach that extends the…

Artificial Intelligence · Computer Science 2024-12-17 Filippo Ziliotto , Tommaso Campari , Luciano Serafini , Lamberto Ballan

Embodied AI aims to develop intelligent systems with physical forms capable of perceiving, decision-making, acting, and learning in real-world environments, providing a promising way to Artificial General Intelligence (AGI). Despite decades…

Robotics · Computer Science 2025-08-15 Wenlong Liang , Rui Zhou , Yang Ma , Bing Zhang , Songlin Li , Yijia Liao , Ping Kuang

Current embodied intelligent systems still face a substantial gap between high-level reasoning and low-level physical execution in open-world environments. Although Vision-Language-Action (VLA) models provide strong perception and intuitive…

Computer Vision and Pattern Recognition · Computer Science 2026-04-20 Dongjie Huo , Haoyun Liu , Guoqing Liu , Dekang Qi , Zhiming Sun , Maoguo Gao , Jianxin He , Yandan Yang , Xinyuan Chang , Feng Xiong , Xing Wei , Zhiheng Ma , Mu Xu

Large language model (LLM)-based agents that reason, plan, and act through tools, memory, and structured interaction are emerging as a promising paradigm for automating complex workflows. Recent systems such as OpenClaw and Claude Code…

Information Retrieval · Computer Science 2026-05-27 Yingli Zhou , Wang Shu , Yaodong Su , Wenchuan Du , Yixiang Fang , Xuemin Lin

This paper introduces LLM-MARL, a unified framework that incorporates large language models (LLMs) into multi-agent reinforcement learning (MARL) to enhance coordination, communication, and generalization in simulated game environments. The…

Artificial Intelligence · Computer Science 2025-11-04 Zhengyang Li , Sawyer Campos , Nana Wang

We propose an agent architecture that automates parts of the common reinforcement learning experiment workflow, to enable automated mastery of control domains for embodied agents. To do so, it leverages a VLM to perform some of the…

Artificial Intelligence · Computer Science 2024-09-06 Jingwei Zhang , Thomas Lampe , Abbas Abdolmaleki , Jost Tobias Springenberg , Martin Riedmiller

The emergence of Large Language Models (LLMs) has reshaped agent systems. Unlike traditional rule-based agents with limited task scope, LLM-powered agents offer greater flexibility, cross-domain reasoning, and natural language interaction.…

Artificial Intelligence · Computer Science 2026-05-05 Guannan Liang , Qianqian Tong

Multi-Agent Reinforcement Learning (MARL) methods have shown promise in enabling agents to learn a shared communication protocol from scratch and accomplish challenging team tasks. However, the learned language is usually not interpretable…

Recent advances in large language models (LLMs) have sparked growing interest in building generalist agents that can learn through online interactions. However, applying reinforcement learning (RL) to train LLM agents in multi-turn,…

Artificial Intelligence · Computer Science 2025-10-07 Hanchen Zhang , Xiao Liu , Bowen Lv , Xueqiao Sun , Bohao Jing , Iat Long Iong , Zhenyu Hou , Zehan Qi , Hanyu Lai , Yifan Xu , Rui Lu , Hongning Wang , Jie Tang , Yuxiao Dong

Large Language Models (LLMs) have revolutionized various aspects of engineering and science. Their utility is often bottlenecked by the lack of interaction with the external digital environment. To overcome this limitation and achieve…

Software Engineering · Computer Science 2024-12-24 Ioannis Tzachristas

Reinforcement learning (RL) agent development traditionally requires substantial expertise and iterative effort, often leading to high failure rates and limited accessibility. This paper introduces Agent$^2$, an LLM-driven…

Artificial Intelligence · Computer Science 2025-10-01 Yuan Wei , Xiaohan Shan , Ran Miao , Jianmin Li

Large Language Model (LLM) agents can leverage tools such as Google Search to complete complex tasks. However, this tool usage introduces the risk of indirect prompt injections, where malicious instructions hidden in tool outputs can…

Machine Learning · Computer Science 2025-10-08 Zizhao Wang , Dingcheng Li , Vaishakh Keshava , Phillip Wallis , Ananth Balashankar , Peter Stone , Lukas Rutishauser

Reinforcement learning (RL) is a popular machine learning paradigm for game playing, robotics control, and other sequential decision tasks. However, RL agents often have long learning times with high data requirements because they begin by…

Machine Learning · Computer Science 2021-02-05 Matthew E. Taylor , Nicholas Nissen , Yuan Wang , Neda Navidi

Open-sourced Large Language Models (LLMs) have achieved great success in various NLP tasks, however, they are still far inferior to API-based models when acting as agents. How to integrate agent ability into general LLMs becomes a crucial…

Computation and Language · Computer Science 2024-03-20 Zehui Chen , Kuikun Liu , Qiuchen Wang , Wenwei Zhang , Jiangning Liu , Dahua Lin , Kai Chen , Feng Zhao

Language is a ubiquitous tool that is foundational to reasoning and collaboration, ranging from everyday interactions to sophisticated problem-solving tasks. The establishment of a common language can serve as a powerful asset in ensuring…

Artificial Intelligence · Computer Science 2025-08-12 Dom Huh , Prasant Mohapatra

Building embodied AI systems that can follow arbitrary language instructions in any 3D environment is a key challenge for creating general AI. Accomplishing this goal requires learning to ground language in perception and embodied actions,…

Robotics · Computer Science 2024-10-14 SIMA Team , Maria Abi Raad , Arun Ahuja , Catarina Barros , Frederic Besse , Andrew Bolt , Adrian Bolton , Bethanie Brownfield , Gavin Buttimore , Max Cant , Sarah Chakera , Stephanie C. Y. Chan , Jeff Clune , Adrian Collister , Vikki Copeman , Alex Cullum , Ishita Dasgupta , Dario de Cesare , Julia Di Trapani , Yani Donchev , Emma Dunleavy , Martin Engelcke , Ryan Faulkner , Frankie Garcia , Charles Gbadamosi , Zhitao Gong , Lucy Gonzales , Kshitij Gupta , Karol Gregor , Arne Olav Hallingstad , Tim Harley , Sam Haves , Felix Hill , Ed Hirst , Drew A. Hudson , Jony Hudson , Steph Hughes-Fitt , Danilo J. Rezende , Mimi Jasarevic , Laura Kampis , Rosemary Ke , Thomas Keck , Junkyung Kim , Oscar Knagg , Kavya Kopparapu , Rory Lawton , Andrew Lampinen , Shane Legg , Alexander Lerchner , Marjorie Limont , Yulan Liu , Maria Loks-Thompson , Joseph Marino , Kathryn Martin Cussons , Loic Matthey , Siobhan Mcloughlin , Piermaria Mendolicchio , Hamza Merzic , Anna Mitenkova , Alexandre Moufarek , Valeria Oliveira , Yanko Oliveira , Hannah Openshaw , Renke Pan , Aneesh Pappu , Alex Platonov , Ollie Purkiss , David Reichert , John Reid , Pierre Harvey Richemond , Tyson Roberts , Giles Ruscoe , Jaume Sanchez Elias , Tasha Sandars , Daniel P. Sawyer , Tim Scholtes , Guy Simmons , Daniel Slater , Hubert Soyer , Heiko Strathmann , Peter Stys , Allison C. Tam , Denis Teplyashin , Tayfun Terzi , Davide Vercelli , Bojan Vujatovic , Marcus Wainwright , Jane X. Wang , Zhengdong Wang , Daan Wierstra , Duncan Williams , Nathaniel Wong , Sarah York , Nick Young

Language models are exhibiting increasing capability in knowledge utilization and reasoning. However, when applied as agents in embodied environments, they often suffer from misalignment between their intrinsic knowledge and environmental…

Computation and Language · Computer Science 2024-10-01 Hanlin Wang , Chak Tou Leong , Jian Wang , Wenjie Li

Embodied AI research is increasingly moving beyond single-task, single-environment policy learning toward multi-task, multi-scene, and multi-model settings. This shift substantially increases the engineering overhead and development time…

Robotics · Computer Science 2026-04-16 Xueyang Zhou , Yihan Sun , Xijie Gong , Guiyao Tie , Pan Zhou , Lichao Sun , Yongchao Chen