English
Related papers

Related papers: Evaluation-Driven Development and Operations of LL…

200 papers

Reinforcement learning algorithms typically struggle in the absence of a dense, well-shaped reward function. Intrinsically motivated exploration methods address this limitation by rewarding agents for visiting novel states or transitions,…

Machine Learning · Computer Science 2023-09-18 Yuqing Du , Olivia Watkins , Zihan Wang , Cédric Colas , Trevor Darrell , Pieter Abbeel , Abhishek Gupta , Jacob Andreas

How to construct an interpretable autonomous driving decision-making system has become a focal point in academic research. In this study, we propose a novel approach that leverages large language models (LLMs) to generate executable,…

Artificial Intelligence · Computer Science 2025-06-18 Fanzhi Zeng , Siqi Wang , Chuzhao Zhu , Li Li

Recent advances in decision-making policies have led to significant progress in fields such as autonomous driving and robotics. However, testing these policies remains crucial with the existence of critical scenarios that may threaten their…

Machine Learning · Computer Science 2024-12-17 Weichao Xu , Huaxin Pei , Jingxuan Yang , Yuchen Shi , Yi Zhang , Qianchuan Zhao

As Large Language Models (LLMs) move from curated training sets into open-ended real-world environments, a fundamental limitation emerges: static training cannot keep pace with continual deployment environment change. Scaling training-time…

Artificial Intelligence · Computer Science 2026-03-17 Minhua Lin , Hanqing Lu , Zhan Shi , Bing He , Rui Mao , Zhiwei Zhang , Zongyu Wu , Xianfeng Tang , Hui Liu , Zhenwei Dai , Xiang Zhang , Suhang Wang , Benoit Dumoulin , Jian Pei

Embodied intelligence empowers agents with a profound sense of perception, enabling them to respond in a manner closely aligned with real-world situations. Large Language Models (LLMs) delve into language instructions with depth, serving a…

Multimedia · Computer Science 2024-07-17 Shuyuan Liu , Jiawei Chen , Shouwei Ruan , Hang Su , Zhaoxia Yin

Context: LLM-based multi-agent systems enable automation and decision support in software development, yet existing studies rely on benchmark datasets offering only binary pass-or-fail results, limiting insight into real-world…

\textit{Background:} The use of large language models in software testing is growing fast as they support numerous tasks, from test case generation to automation, and documentation. However, their adoption often relies on informal…

Software Engineering · Computer Science 2025-10-21 Maria Deolinda Santana , Cleyton Magalhaes , Ronnie de Souza Santos

Large Language Models (LLMs) are increasingly utilized for domain-specific tasks, yet evaluating their outputs remains challenging. A common strategy is to apply evaluation criteria to assess alignment with domain-specific standards, yet…

Human-Computer Interaction · Computer Science 2026-02-17 Annalisa Szymanski , Simret Araya Gebreegziabher , Oghenemaro Anuyah , Ronald A. Metoyer , Toby Jia-Jun Li

As AI continues to advance, there is a growing demand for systems that go beyond language-based assistance and move toward intelligent agents capable of performing real-world actions. This evolution requires the transition from traditional…

With the growing adoption of Large Language Models (LLMs) in automating complex, multi-agent workflows, organizations face mounting risks from errors, emergent behaviors, and systemic failures that current evaluation methods fail to…

Artificial Intelligence · Computer Science 2025-09-19 NVJK Kartik , Garvit Sapra , Rishav Hada , Nikhil Pareek

Large Language Model (LLM)-based Multi-Agent Systems (MAS) enable complex problem-solving but introduce significant debugging challenges, characterized by long interaction traces, inter-agent dependencies, and delayed error manifestation.…

Multiagent Systems · Computer Science 2026-04-21 Jiazheng Li , Emine Yilmaz , Bei Chen , Dieu-Thu Le

In recent years, Large Language Models (LLMs) have shown great abilities in various tasks, including question answering, arithmetic problem solving, and poem writing, among others. Although research on LLM-as-an-agent has shown that LLM can…

Multiagent Systems · Computer Science 2024-05-21 Chuanneng Sun , Songjun Huang , Dario Pompili

Large Language Model (LLM)-based multi-agent systems (MAS) have emerged as a promising paradigm for solving complex tasks. However, existing works often rely on manual designs or "one-size-fits-all" automation, lacking dynamic adaptability…

Multiagent Systems · Computer Science 2026-02-17 Guangyi Liu , Haojun Lin , Huan Zeng , Heng Wang , Quanming Yao

Large language models (LLMs) have revolutionized natural language processing with their exceptional understanding, synthesizing, and reasoning capabilities. However, deploying LLMs on resource-constrained edge devices presents significant…

Distributed, Parallel, and Cluster Computing · Computer Science 2025-02-25 Yue Zheng , Yuhao Chen , Bin Qian , Xiufang Shi , Yuanchao Shu , Jiming Chen

While large language models (LMs) have shown remarkable capabilities across numerous tasks, they often struggle with simple reasoning and planning in physical environments, such as understanding object permanence or planning household…

Computation and Language · Computer Science 2023-10-31 Jiannan Xiang , Tianhua Tao , Yi Gu , Tianmin Shu , Zirui Wang , Zichao Yang , Zhiting Hu

The rapid progress of Large Language Models (LLMs) has given rise to a new category of autonomous AI systems, referred to as Deep Research (DR) agents. These agents are designed to tackle complex, multi-turn informational research tasks by…

Artificial Intelligence · Computer Science 2025-09-04 Yuxuan Huang , Yihang Chen , Haozheng Zhang , Kang Li , Huichi Zhou , Meng Fang , Linyi Yang , Xiaoguang Li , Lifeng Shang , Songcen Xu , Jianye Hao , Kun Shao , Jun Wang

Tool-using agents based on Large Language Models (LLMs) excel in tasks such as mathematical reasoning and multi-hop question answering. However, in long trajectories, agents often trigger excessive and low-quality tool calls, increasing…

Artificial Intelligence · Computer Science 2026-03-25 Zeping Li , Hongru Wang , Yiwen Zhao , Guanhua Chen , Yixia Li , Keyang Chen , Yixin Cao , Guangnan Ye , Hongfeng Chai , Zhenfei Yin

The integration of Large Language Models (LLMs) into software engineering has driven a transition from traditional rule-based systems to autonomous agentic systems capable of solving complex problems. However, systematic progress is…

Software Engineering · Computer Science 2025-10-24 Jiale Guo , Suizhi Huang , Mei Li , Dong Huang , Xingsheng Chen , Regina Zhang , Zhijiang Guo , Han Yu , Siu-Ming Yiu , Pietro Lio , Kwok-Yan Lam

Large Language Models (LLMs) and Reinforcement Learning (RL) are two powerful approaches for building autonomous agents. However, due to limited understanding of the game environment, agents often resort to inefficient exploration and…

Machine Learning · Computer Science 2024-11-26 Ziyu Chen , Zhiqing Xiao , Xinbei Jiang , Junbo Zhao

Large Language Models (LLMs) exhibit robust problem-solving capabilities for diverse tasks. However, most LLM-based agents are designed as specific task solvers with sophisticated prompt engineering, rather than agents capable of learning…

Artificial Intelligence · Computer Science 2024-06-10 Wenqi Zhang , Ke Tang , Hai Wu , Mengna Wang , Yongliang Shen , Guiyang Hou , Zeqi Tan , Peng Li , Yueting Zhuang , Weiming Lu