English
Related papers

Related papers: Luban: Building Open-Ended Creative Agents via Aut…

200 papers

While large language models (LLMs) have shown promising capabilities as zero-shot planners for embodied agents, their inability to learn from experience and build persistent mental models limits their robustness in complex open-world…

Artificial Intelligence · Computer Science 2025-06-04 Anirudh Chari , Suraj Reddy , Aditya Tiwari , Richard Lian , Brian Zhou

As agentic AI systems increasingly operate autonomously, establishing trust through verifiable evaluation becomes critical. Yet existing benchmarks lack the transparency and auditability needed to assess whether agents behave reliably. We…

Computation and Language · Computer Science 2025-12-02 Hyunjun Kim , Sooyoung Ryu

While Vision-Language Models (VLMs) hold promise for tasks requiring extensive collaboration, traditional multi-agent simulators have facilitated rich explorations of an interactive artificial society that reflects collective behavior.…

Computation and Language · Computer Science 2024-05-24 Xianhao Yu , Jiaqi Fu , Renjia Deng , Wenjuan Han

Multimodal Large Language Models (MLLMs) are evolving from passive observers into active agents, solving problems through Visual Expansion (invoking visual tools) and Knowledge Expansion (open-web search). However, existing evaluations fall…

Artificial Intelligence · Computer Science 2026-04-06 Qianshan Wei , Yishan Yang , Siyi Wang , Jinglin Chen , Binyu Wang , Jiaming Wang , Shuang Chen , Zechen Li , Yang Shi , Yuqi Tang , Weining Wang , Yi Yu , Chaoyou Fu , Qi Li , Yi-Fan Zhang

Automatically compiling open-source software (OSS) projects is a vital, labor-intensive, and complex task, which makes it a good challenge for LLM Agents. Existing methods rely on manually curated rules and workflows, which cannot adapt to…

Developing autonomous home robots controlled by natural language has long been a pursuit of humanity. While advancements in large language models (LLMs) and embodied intelligence make this goal closer, several challenges persist: the lack…

Robotics · Computer Science 2025-05-16 Dongping Li , Tielong Cai , Tianci Tang , Wenhao Chai , Katherine Rose Driggs-Campbell , Gaoang Wang

Large language models (LLMs) demonstrate strong potential as agents for tool invocation due to their advanced comprehension and planning capabilities. Users increasingly rely on LLM-based agents to solve complex missions through iterative…

Artificial Intelligence · Computer Science 2025-04-17 Peijie Yu , Yifan Yang , Jinjian Li , Zelong Zhang , Haorui Wang , Xiao Feng , Feng Zhang

As large language models (LLMs) continue to improve in reasoning and decision-making, there is a growing need for realistic and interactive environments where their abilities can be rigorously evaluated. We present VirtualEnv, a…

Artificial Intelligence · Computer Science 2026-02-10 Kabir Swain , Sijie Han , Ayush Raina , Jin Zhang , Shuang Li , Michael Stopa , Antonio Torralba

LLM/VLM-based digital agents have advanced rapidly thanks to scalable sandboxes for coding, web navigation, and computer use, which provide rich interactive training grounds. In contrast, embodied agents still lack abundant, diverse, and…

Artificial Intelligence · Computer Science 2026-05-14 Haoqiang Kang , Xiaokang Ye , Yuhan Liu , Siddhant Hitesh Mantri , Lingjun Mao , James Fleming , Drishti Regmi , Lianhui Qin

Existing text-to-image models still struggle to generate images of multiple objects, especially in handling their spatial positions, relative sizes, overlapping, and attribute bindings. To efficiently address these challenges, we develop a…

Computer Vision and Pattern Recognition · Computer Science 2024-05-27 Sen Li , Ruochen Wang , Cho-Jui Hsieh , Minhao Cheng , Tianyi Zhou

Incentivized exploration in multi-armed bandits (MAB) has witnessed increasing interests and many progresses in recent years, where a principal offers bonuses to agents to do explorations on her behalf. However, almost all existing studies…

Machine Learning · Statistics 2021-10-28 Chengshuai Shi , Haifeng Xu , Wei Xiong , Cong Shen

Reinforcement learning with verifiable rewards (RLVR) is pivotal for the continuous evolution of GUI agents, yet existing evaluation paradigms face significant limitations. Rule-based methods suffer from poor scalability and cannot handle…

Robotics · Computer Science 2026-02-03 Chaoqun Cui , Jing Huang , Shijing Wang , Liming Zheng , Qingchao Kong , Zhixiong Zeng

Recent advancements in Large Language Models (LLMs) have not only showcased impressive creative capabilities but also revealed emerging agentic behaviors that exploit linguistic ambiguity in adversarial settings. In this study, we…

Computation and Language · Computer Science 2025-04-04 Seunghyun Yoo

Autonomous agents have recently achieved remarkable progress across diverse domains, yet most evaluations focus on short-horizon, fully observable tasks. In contrast, many critical real-world tasks, such as large-scale software development,…

Advances in large models, reinforcement learning, and open-endedness have accelerated progress toward autonomous agents that can learn and interact in the real world. To achieve this, flexible tools are needed to create rich, yet…

Artificial Intelligence · Computer Science 2025-06-05 Mikel Malagón , Josu Ceberio , Jose A. Lozano

Humans can perform complex tasks with long-term objectives by planning, reasoning, and forecasting outcomes of actions. For embodied agents to achieve similar capabilities, they must gain knowledge of the environment transferable to novel…

Machine Learning · Computer Science 2024-10-01 Shu Ishida

Embodied task planning requires agents to execute long-horizon, goal-directed actions in complex 3D environments, where success depends on both immediate perception and accumulated experience across tasks. However, most existing LLM-based…

Robotics · Computer Science 2026-04-21 Xiaoyu Ma , Lianyu Hu , Wenbing Tang , Zixuan Hu , Zeqin Liao , Zhizhen Wu , Yang Liu

The growing dependence on mobile phones and their apps has made multi-user interactive features, like chat calls, live streaming, and video conferencing, indispensable for bridging the gaps in social connectivity caused by physical and…

Software Engineering · Computer Science 2025-09-17 Sidong Feng , Changhao Du , Huaxiao Liu , Qingnan Wang , Zhengwei Lv , Mengfei Wang , Chunyang Chen

Large Language Models (LLMs) based autonomous agents demonstrate multifaceted capabilities to contribute substantially to economic production. However, existing benchmarks remain focused on single agentic capability, failing to capture…

Artificial Intelligence · Computer Science 2026-04-24 Keyu Li , Junhao Shi , Yang Xiao , Mohan Jiang , Jie Sun , Yunze Wu , Dayuan Fu , Shijie Xia , Xiaojie Cai , Tianze Xu , Weiye Si , Wenjie Li , Dequan Wang , Pengfei Liu

It is a long-lasting goal to design an embodied system that can solve long-horizon open-world tasks in human-like ways. However, existing approaches usually struggle with compound difficulties caused by the logic-aware decomposition and…

Computer Vision and Pattern Recognition · Computer Science 2024-03-28 Yiran Qin , Enshen Zhou , Qichang Liu , Zhenfei Yin , Lu Sheng , Ruimao Zhang , Yu Qiao , Jing Shao