English
Related papers

Related papers: Survey on Evaluation of LLM-based Agents

200 papers

This paper examines the evolution, architecture, and practical applications of AI agents from their early, rule-based incarnations to modern sophisticated systems that integrate large language models with dedicated modules for perception,…

Artificial Intelligence · Computer Science 2025-03-18 Naveen Krishnan

AI agents are systems capable of perceiving their environment, autonomously planning and executing tasks. Recent advancements in LLM have introduced a transformative paradigm for AI agents, enabling them to interact with external resources…

Software Engineering · Computer Science 2024-12-30 Kaiwen Ning , Jiachi Chen , Jingwen Zhang , Wei Li , Zexu Wang , Yuming Feng , Weizhe Zhang , Zibin Zheng

Recent advances in multi-modal large language models (MLLMs) have enabled increasingly sophisticated autonomous visualization agents capable of translating user intentions into data visualizations. However, measuring progress and comparing…

Human-Computer Interaction · Computer Science 2025-09-19 Kuangshi Ai , Haichao Miao , Zhimin Li , Chaoli Wang , Shusen Liu

Artificial Intelligence is moving from models that only generate text to Agentic AI, where systems behave as autonomous entities that can perceive, reason, plan, and act. Large Language Models (LLMs) are no longer used only as passive…

Artificial Intelligence · Computer Science 2026-01-21 Arunkumar V , Gangadharan G. R. , Rajkumar Buyya

Peer review is fundamental to the integrity and advancement of scientific publication. Traditional methods of peer review analyses often rely on exploration and statistics of existing peer review data, which do not adequately address the…

Computation and Language · Computer Science 2026-05-12 Yiqiao Jin , Qinlin Zhao , Yiyang Wang , Hao Chen , Kaijie Zhu , Yijia Xiao , Jindong Wang

The rise of Agent AI and Large Language Model-powered Multi-Agent Systems (LLM-MAS) has underscored the need for responsible and dependable system operation. Tools like LangChain and Retrieval-Augmented Generation have expanded LLM…

Multiagent Systems · Computer Science 2025-02-05 Jinwei Hu , Yi Dong , Shuang Ao , Zhuoyun Li , Boxuan Wang , Lokesh Singh , Guangliang Cheng , Sarvapali D. Ramchurn , Xiaowei Huang

DeepResearch agents represent a transformative AI paradigm, conducting expert-level research through sophisticated reasoning and multi-tool integration. However, evaluating these systems remains critically challenging due to open-ended…

Artificial Intelligence · Computer Science 2025-10-10 Tianyu Fan , Xinyao Niu , Yuxiang Zheng , Fengji Zhang , Chengen Huang , Bei Chen , Junyang Lin , Chao Huang

AI agents -- systems that combine foundation models with reasoning, planning, memory, and tool use -- are rapidly becoming a practical interface between natural-language intent and real-world computation. This survey synthesizes the…

Artificial Intelligence · Computer Science 2026-01-06 Bin Xu

Developing autonomous agents for web-based tasks is a core challenge in AI. While Large Language Model (LLM) agents can interpret complex user requests, they often operate as black boxes, making it difficult to diagnose why they fail or how…

Artificial Intelligence · Computer Science 2026-03-16 Orit Shahnovsky , Rotem Dror

Large Language Models (LLMs) have achieved remarkable success across a wide array of tasks. Due to the impressive planning and reasoning abilities of LLMs, they have been used as autonomous agents to do many tasks automatically. Recently,…

Computation and Language · Computer Science 2024-04-22 Taicheng Guo , Xiuying Chen , Yaqi Wang , Ruidi Chang , Shichao Pei , Nitesh V. Chawla , Olaf Wiest , Xiangliang Zhang

Large language models (LLMs) have sparked growing interest in machine learning research agents that can autonomously propose ideas and conduct experiments. However, existing benchmarks predominantly adopt an engineering-oriented…

Computation and Language · Computer Science 2026-02-26 Qiran Zou , Hou Hei Lam , Wenhao Zhao , Yiming Tang , Tingting Chen , Samson Yu , Tianyi Zhang , Chang Liu , Xiangyang Ji , Dianbo Liu

Large Language Models (LLMs) are increasingly deployed within agentic systems - collections of interacting, LLM-powered agents that execute complex, adaptive workflows using memory, tools, and dynamic planning. While enabling powerful new…

Artificial Intelligence · Computer Science 2025-11-21 Dany Moshkovich , Sergey Zeltyn

Large Language Model (LLM) agents are transforming education by automating complex pedagogical tasks and enhancing both teaching and learning processes. In this survey, we present a systematic review of recent advances in applying LLM…

Computers and Society · Computer Science 2026-02-05 Zhendong Chu , Shen Wang , Jian Xie , Tinghui Zhu , Yibo Yan , Jinheng Ye , Aoxiao Zhong , Xuming Hu , Jing Liang , Philip S. Yu , Qingsong Wen

The convergence of artificial intelligence and materials science presents a transformative opportunity, but achieving true acceleration in discovery requires moving beyond task-isolated, fine-tuned models toward agentic systems that plan,…

Since the advent of Large Language Models (LLMs), various research based on such models have maintained significant academic attention and impact, especially in AI and robotics. In this paper, we propose a multi-agent framework with LLMs to…

Robotics · Computer Science 2025-05-12 Junhong Chen , Ziqi Yang , Haoyuan G Xu , Dandan Zhang , George Mylonas

LLM-based Multi-Agent Systems ( LLM-MAS ) have become a research hotspot since the rise of large language models (LLMs). However, with the continuous influx of new related works, the existing reviews struggle to capture them…

Computation and Language · Computer Science 2025-01-08 Shuaihang Chen , Yuanxing Liu , Wei Han , Weinan Zhang , Ting Liu

Large language model (LLM)-driven multi-agent systems (MAS) are transforming how humans and AIs collaboratively generate ideas and artifacts. While existing surveys provide comprehensive overviews of MAS infrastructures, they largely…

Human-Computer Interaction · Computer Science 2025-05-28 Yi-Cheng Lin , Kang-Chieh Chen , Zhe-Yan Li , Tzu-Heng Wu , Tzu-Hsuan Wu , Kuan-Yu Chen , Hung-yi Lee , Yun-Nung Chen

Agentic systems have transformed how Large Language Models (LLMs) can be leveraged to create autonomous systems with goal-directed behaviors, consisting of multi-step planning and the ability to interact with different environments. These…

Artificial Intelligence · Computer Science 2026-01-27 Judy Zhu , Dhari Gandhi , Himanshu Joshi , Ahmad Rezaie Mianroodi , Sedef Akinli Kocak , Dhanesh Ramachandran

Large language model (LLM)-based agents are increasingly applied to complex strategic environments that demand long-horizon reasoning, multi-agent interaction, and decision-making under uncertainty. However, common existing benchmarks…

Artificial Intelligence · Computer Science 2026-05-12 Wenjie Tang , Yuan Zhou , Erqiang Xu , Keyan Cheng , Minne Li , Liquan Xiao

Large language models (LLMs) are gaining increasing interests to improve clinical efficiency for medical diagnosis, owing to their unprecedented performance in modelling natural language. Ensuring the safe and reliable clinical…

Computation and Language · Computer Science 2024-03-26 Lei Liu , Xiaoyan Yang , Fangzhou Li , Chenfei Chi , Yue Shen , Shiwei Lyu Ming Zhang , Xiaowei Ma , Xiangguo Lyu , Liya Ma , Zhiqiang Zhang , Wei Xue , Yiran Huang , Jinjie Gu