English
Related papers

Related papers: Luban: Building Open-Ended Creative Agents via Aut…

200 papers

LLM-based agents are increasingly expected to handle real-world assistant tasks, yet existing benchmarks typically evaluate them under isolated sources of difficulty, such as a single environment or fully specified instructions. This leaves…

Computation and Language · Computer Science 2026-04-16 Xiang Long , Li Du , Yilong Xu , Fangcheng Liu , Haoqing Wang , Ning Ding , Ziheng Li , Jianyuan Guo , Yehui Tang

Building an embodied agent system with a large language model (LLM) as its core is a promising direction. Due to the significant costs and uncontrollable factors associated with deploying and training such agents in the real world, we have…

Computer Vision and Pattern Recognition · Computer Science 2024-06-18 Zhonghan Zhao , Wenhao Chai , Xuan Wang , Ke Ma , Kewei Chen , Dongxu Guo , Tian Ye , Yanting Zhang , Hongwei Wang , Gaoang Wang

We present PEAM, a Parametric Embodied Agent Memory framework in Minecraft that transforms agent memory from inference-time retrieval into parameter-resident skills internalized through experience. PEAM pairs a slow deliberative LLM for…

Artificial Intelligence · Computer Science 2026-05-28 Yuchen Guo , Junli Gong , Hongmin Cai , Yiu-ming Cheung , Weifeng Su

Recent advancements in Large Language Models (LLMs) have enhanced the reasoning capabilities of embodied agents, driving progress toward AGI-powered robotics. While LLMs have been applied to tasks like semantic reasoning and task…

Human-Computer Interaction · Computer Science 2024-12-31 Jia Liu , Min Chen

Embodied computer vision considers perception for robots in novel, unstructured environments. Of particular importance is the embodied visual exploration problem: how might a robot equipped with a camera scope out a new environment? Despite…

Computer Vision and Pattern Recognition · Computer Science 2020-08-24 Santhosh K. Ramakrishnan , Dinesh Jayaraman , Kristen Grauman

We present app.build (https://github.com/neondatabase/appdotbuild-agent), an open-source framework that improves LLM-based application generation through systematic validation and structured environments. Our approach combines multi-layered…

Artificial Intelligence · Computer Science 2026-01-13 Evgenii Kniazev , Arseny Kravchenko , Igor Rekun , James Broadhead , Nikita Shamgunov , Pranav Sah , Pratik Nichite , Ivan Yamshchikov

This paper reviews the architecture and implementation methods of agents powered by large language models (LLMs). Motivated by the limitations of traditional LLMs in real-world tasks, the research aims to explore patterns to develop…

Artificial Intelligence · Computer Science 2025-10-13 Victor de Lamo Castrillo , Habtom Kahsay Gidey , Alexander Lenz , Alois Knoll

As multimodal large language models (MLLMs) advance, MLLM-based virtual agents have demonstrated remarkable performance. However, existing benchmarks face significant limitations, including uncontrollable task complexity, extensive manual…

Computer Vision and Pattern Recognition · Computer Science 2025-06-11 Wendong Bu , Yang Wu , Qifan Yu , Minghe Gao , Bingchen Miao , Zhenkui Zhang , Kaihang Pan , Yunfei Li , Mengze Li , Wei Ji , Juncheng Li , Siliang Tang , Yueting Zhuang

Automated code vulnerability detection is critical for software security, yet existing approaches face a fundamental trade-off between detection accuracy and computational cost. We propose a heterogeneous multi-agent architecture inspired…

Cryptography and Security · Computer Science 2026-04-24 Zhaohui Geoffrey Wang

Modern video games pose significant challenges for traditional automated testing algorithms, yet intensive testing is crucial to ensure game quality. To address these challenges, researchers designed gaming agents using Reinforcement…

Software Engineering · Computer Science 2026-02-23 Yifei Chen , Sarra Habchi , Lili Wei

The development of autonomous agents increasingly relies on Multimodal Language Models (MLMs) to perform tasks described in natural language with GUI environments, such as websites, desktop computers, or mobile phones. Existing benchmarks…

With the recent development of natural language generation models - termed as large language models (LLMs) - a potential use case has opened up to improve the way that humans interact with robot assistants. These LLMs should be able to…

Multiagent Systems · Computer Science 2024-11-27 Mitchell Rosser , Marc. G Carmichael

We introduce CollabToolBuilder, a flexible multiagent LLM framework with expert-in-the-loop (HITL) guidance that iteratively learns to create tools for a target goal, aligning with human intent and process, while minimizing time for…

Artificial Intelligence · Computer Science 2025-12-02 Daull Xavier , Patrice Bellot , Emmanuel Bruno , Vincent Martin , Elisabeth Murisasco

Embodied exploration is a target-driven process that requires embodied agents to possess fine-grained perception and knowledge-enhanced decision making. While recent attempts leverage MLLMs for exploration due to their strong perceptual and…

Computer Vision and Pattern Recognition · Computer Science 2025-11-25 Gengyuan Zhang , Mingcong Ding , Jingpei Wu , Ruotong Liao , Volker Tresp

Digital human generation has been studied for decades and supports a wide range of real-world applications. However, most existing systems are passively animated, relying on privileged state or scripted control, which limits scalability to…

Computer Vision and Pattern Recognition · Computer Science 2026-04-10 Hang Ye , Xiaoxuan Ma , Fan Lu , Wayne Wu , Kwan-Yee Lin , Yizhou Wang

Humans, even at a very early age, can learn visual concepts and understand geometry and layout through active interaction with the environment, and generalize their compositions to complete tasks described by natural languages in novel…

Computer Vision and Pattern Recognition · Computer Science 2023-04-10 Mingyu Ding , Yan Xu , Zhenfang Chen , David Daniel Cox , Ping Luo , Joshua B. Tenenbaum , Chuang Gan

The widespread adoption of AI-assisted development tools in 2025 -- and the emergence of vibe coding, a practice of generating complete applications from natural language without verification -- exposed a critical and tool-agnostic failure…

Software Engineering · Computer Science 2026-05-01 Jasmine Moreira

Creativity is a fundamental aspect of intelligence, involving the ability to generate novel and appropriate solutions across diverse contexts. While Large Language Models (LLMs) have been extensively evaluated for their creative…

Computer Vision and Pattern Recognition · Computer Science 2025-03-20 Xinyu Fang , Zhijian Chen , Kai Lan , Lixin Ma , Shengyuan Ding , Yingji Liang , Xiangyu Zhao , Farong Wen , Zicheng Zhang , Guofeng Zhang , Haodong Duan , Kai Chen , Dahua Lin

As embodied intelligence emerges as a core frontier in artificial intelligence research, simulation platforms must evolve beyond low-level physical interactions to capture complex, human-centered social behaviors. We introduce FreeAskWorld,…

Artificial Intelligence · Computer Science 2025-12-23 Yuhang Peng , Yizhou Pan , Xinning He , Jihaoyu Yang , Xinyu Yin , Han Wang , Xiaoji Zheng , Chao Gao , Jiangtao Gong

In the realm of computer vision and robotics, embodied agents are expected to explore their environment and carry out human instructions. This necessitates the ability to fully understand 3D scenes given their first-person observations and…

Computer Vision and Pattern Recognition · Computer Science 2023-12-27 Tai Wang , Xiaohan Mao , Chenming Zhu , Runsen Xu , Ruiyuan Lyu , Peisen Li , Xiao Chen , Wenwei Zhang , Kai Chen , Tianfan Xue , Xihui Liu , Cewu Lu , Dahua Lin , Jiangmiao Pang