中文
相关论文

相关论文: VeRO: An Evaluation Harness for Agents to Optimize…

200 篇论文

Hindsight Experience Replay (HER) is a multi-goal reinforcement learning algorithm for sparse reward functions. The algorithm treats every failure as a success for an alternative (virtual) goal that has been achieved in the episode. Virtual…

机器学习 · 计算机科学 2021-03-09 Binyamin Manela , Armin Biess

A common challenge in reinforcement learning is how to convert the agent's interactions with an environment into fast and robust learning. For instance, earlier work makes use of domain knowledge to improve existing reinforcement learning…

机器学习 · 计算机科学 2020-04-01 Yannis Flet-Berliac , Philippe Preux

The rapid adoption of AI agents across domains has made systematic evaluation crucial for ensuring their usefulness and successful production deployment. Evaluation of AI agents typically involves using a fixed set of benchmarks and…

Long-horizon decision-making tasks present significant challenges for LLM-based agents due to the need for extensive planning over multiple steps. In this paper, we propose a hierarchical framework that decomposes complex tasks into…

机器学习 · 计算机科学 2024-10-07 Qi Zhao , Haotian Fu , Chen Sun , George Konidaris

In order for agents trained by deep reinforcement learning to work alongside humans in realistic settings, we will need to ensure that the agents are \emph{robust}. Since the real world is very diverse, and human behavior often changes in…

机器学习 · 计算机科学 2021-01-15 Paul Knott , Micah Carroll , Sam Devlin , Kamil Ciosek , Katja Hofmann , A. D. Dragan , Rohin Shah

Generative auto-bidding has demonstrated strong performance in online advertising, yet it often suffers from data scarcity in small-scale settings with limited advertiser participation. While cross-task data sharing is a natural remedy to…

机器学习 · 计算机科学 2026-02-10 Yiqin Lv , Zhiyu Mou , Miao Xu , Jinghao Chen , Qi Wang , Yixiu Mao , Yun Qu , Rongquan Bai , Chuan Yu , Jian Xu , Bo Zheng , Xiangyang Ji

Agent-based IoT applications have recently been proposed in several domains, such as health care, smart cities and agriculture. Deploying these applications in specific settings has been very challenging for many reasons including the…

多智能体系统 · 计算机科学 2018-02-13 Nathalia Nascimento , Paulo Alencar , Carlos Lucena , Donald Cowan

Learning to solve long horizon temporally extended tasks with reinforcement learning has been a challenge for several years now. We believe that it is important to leverage both the hierarchical structure of complex tasks and to use expert…

机器学习 · 计算机科学 2022-10-18 Bharat Prakash , Nicholas Waytowich , Tim Oates , Tinoosh Mohsenin

Recent studies show that LLMs possess different skills and specialize in different tasks. In fact, we observe that their varied performance occur in several levels of granularity. For example, in the code optimization task, code LLMs excel…

人工智能 · 计算机科学 2025-10-24 Yuanzhe Liu , Ryan Deng , Tim Kaler , Xuhao Chen , Charles E. Leiserson , Yao Ma , Jie Chen

Software testing framework can be stated as the process of verifying and validating that a computer program/application works as expected and meets the requirements of the user. Usually testing can be done manually or using tools. Manual…

软件工程 · 计算机科学 2013-07-15 K. Karnavel , V. Divya , Gnanakeerthika , P. Karthika

Coding agents are increasingly used as iterative development partners, but most benchmarks still evaluate one specification followed by one final assessment. This leaves out a basic question: can an agent keep its own codebase working as…

人工智能 · 计算机科学 2026-05-26 Haiyang Shen , Xuanzhong Chen , Wendong Xu , Yun Ma , Liang Chen , Kuan Li

LLM agents increasingly perform end-to-end ML engineering tasks where success is judged by a single scalar test metric. This creates a structural vulnerability: an agent can increase the reported score by compromising the evaluation…

人工智能 · 计算机科学 2026-03-13 Yonas Atinafu , Robin Cohen

Agent harness evolution improves frozen language-model agents by modifying the executable structures around them. We study this paradigm as a form of sample-efficient fast adaptation: instead of updating model weights, an agent can acquire…

人工智能 · 计算机科学 2026-05-26 Lirong Che , Yuzhe yang , Peiwen lin , Chuang wang , Xueqian wang , Jian su

The advent of large language models trained on code (code LLMs) has led to significant progress in language-to-code generation. State-of-the-art approaches in this area combine LLM decoding with sample pruning and reranking using test cases…

机器学习 · 计算机科学 2023-09-04 Ansong Ni , Srini Iyer , Dragomir Radev , Ves Stoyanov , Wen-tau Yih , Sida I. Wang , Xi Victoria Lin

Reinforcement learning (RL) agent development traditionally requires substantial expertise and iterative effort, often leading to high failure rates and limited accessibility. This paper introduces Agent$^2$, an LLM-driven…

人工智能 · 计算机科学 2025-10-01 Yuan Wei , Xiaohan Shan , Ran Miao , Jianmin Li

Although multi-agent systems based on large language models show strong capabilities on multiple tasks, they are still limited by high computational overhead, information loss, and robustness. Inspired by ResNet's residual learning, we…

人工智能 · 计算机科学 2025-06-02 Zhentao Xie , Chengcheng Han , Jinxin Shi , Wenjun Cui , Xin Zhao , Xingjiao Wu , Jiabao Zhao

Conventional visual object trackers localize targets using handcrafted spatial priors, often in the form of heatmaps. Such priors provide only surrogate supervision and are poorly aligned with tracking optimization and evaluation metrics,…

计算机视觉与模式识别 · 计算机科学 2026-05-20 Xin Chen , Chuanyu Sun , Jiao Xu , Houwen Peng , Dong Wang , Huchuan Lu , Kede Ma

Pre-trained Vision-Language Models (VLMs) are able to understand visual concepts, describe and decompose complex tasks into sub-tasks, and provide feedback on task completion. In this paper, we aim to leverage these capabilities to support…

机器学习 · 计算机科学 2024-02-08 David Venuto , Sami Nur Islam , Martin Klissarov , Doina Precup , Sherry Yang , Ankit Anand

Automatic multi-agent systems aim to instantiate agent workflows without relying on manually designed or fixed orchestration. However, existing automatic MAS approaches remain only partially adaptive: they either perform training-free…

人工智能 · 计算机科学 2026-05-15 Yaolun Zhang , Yujie Zhao , Nan Wang , Yiran Wu , Jiayu Chang , Yizhao Chen , Qingyun Wu , Jishen Zhao , Huazheng Wang

The development of LLM-based autonomous agents for end-to-end software development represents a significant paradigm shift in software engineering. However, the scientific evaluation of these systems is hampered by significant challenges,…

软件工程 · 计算机科学 2025-11-07 Zhengran Zeng , Yixin Li , Rui Xie , Wei Ye , Shikun Zhang