English
Related papers

Related papers: VeRO: An Evaluation Harness for Agents to Optimize…

200 papers

In many reinforcement learning (RL) applications, augmenting the task rewards with heuristic rewards that encode human priors about how a task should be solved is crucial for achieving desirable performance. However, because such heuristics…

Machine Learning · Computer Science 2025-07-09 Chi-Chang Lee , Zhang-Wei Hong , Pulkit Agrawal

Large language models (LLMs) have transformed software development through code generation capabilities, yet their effectiveness for high-performance computing (HPC) remains limited. HPC code requires specialized optimizations for…

Distributed, Parallel, and Cluster Computing · Computer Science 2025-06-26 Asif Rahman , Veljko Cvetkovic , Kathleen Reece , Aidan Walters , Yasir Hassan , Aneesh Tummeti , Bryan Torres , Denise Cooney , Margaret Ellis , Dimitrios S. Nikolopoulos

As large language model agents tackle increasingly complex long-horizon tasks, effective post-training becomes critical. Prior work faces fundamental challenges: outcome-only rewards fail to precisely attribute credit to intermediate steps,…

Computation and Language · Computer Science 2026-04-30 Mukai Li , Qingcheng Zeng , Tianqing Fang , Zhenwen Liang , Linfeng Song , Qi Liu , Haitao Mi , Dong Yu

Reinforcement learning with verifiable rewards (RLVR) has advanced the reasoning capabilities of large language models. However, existing methods rely solely on outcome rewards, without explicitly optimizing verification or leveraging…

Software Engineering · Computer Science 2025-10-22 Yiyang Jin , Kunzhao Xu , Hang Li , Xueting Han , Yanmin Zhou , Cheng Li , Jing Bai

The performance of large language model (LLM) systems depends not only on model weights, but also on their harness: the code that determines what information to store, retrieve, and present to the model. Yet harnesses are still designed…

Artificial Intelligence · Computer Science 2026-03-31 Yoonho Lee , Roshen Nair , Qizheng Zhang , Kangwook Lee , Omar Khattab , Chelsea Finn

Modern AI agents optimize programs by refactoring source code to trigger trusted compiler transformations. This preserves program semantics and reduces source code pollution, making the program easier to maintain and portable across…

Programming Languages · Computer Science 2026-04-16 Akash Deo , Simone Campanoni , Tommy McMichen

Reinforcement Learning (RL) in games has gained significant momentum in recent years, enabling the creation of different agent behaviors that can transform a player's gaming experience. However, deploying RL agents in production…

Artificial Intelligence · Computer Science 2025-07-01 António Afonso , Iolanda Leite , Alessandro Sestini , Florian Fuchs , Konrad Tollmar , Linus Gisslén

LLM-based optimization has shown remarkable potential in enhancing agentic systems. However, the conventional approach of prompting LLM optimizer with the whole training trajectories on training dataset in a single pass becomes untenable as…

Computation and Language · Computer Science 2025-05-08 Jiale Liu , Yifan Zeng , Shaokun Zhang , Chi Zhang , Malte Højmark-Bertelsen , Marie Normann Gadeberg , Huazheng Wang , Qingyun Wu

Visual Odometry (VO) is essential to downstream mobile robotics and augmented/virtual reality tasks. Despite recent advances, existing VO methods still rely on heuristic design choices that require several weeks of hyperparameter tuning by…

Computer Vision and Pattern Recognition · Computer Science 2024-07-23 Nico Messikommer , Giovanni Cioffi , Mathias Gehrig , Davide Scaramuzza

Modern language models often need to optimize a primary accuracy objective while also accommodating secondary behavioral preferences, such as verbosity, agreeableness, or the level of technical expertise in its response. In practice, a base…

Machine Learning · Computer Science 2026-05-18 Xuechen Zhang , Zijian Huang , Kai Yang , Weijia Zhang , Jiasi Chen , Samet Oymak

The increasing use of Retrieval-Augmented Generation (RAG) systems in various applications necessitates stringent protocols to ensure RAG systems accuracy, safety, and alignment with user intentions. In this paper, we introduce VERA…

Information Retrieval · Computer Science 2024-09-09 Tianyu Ding , Adi Banerjee , Laurent Mombaerts , Yunhong Li , Tarik Borogovac , Juan Pablo De la Cruz Weinstein

Language Models and Vision Language Models have recently demonstrated unprecedented capabilities in terms of understanding human intentions, reasoning, scene understanding, and planning-like behaviour, in text form, among many others. In…

As intelligent agents become more generally-capable, i.e. able to master a wide variety of tasks, the complexity and cost of properly evaluating them rises significantly. Tasks that assess specific capabilities of the agents can be…

Artificial Intelligence · Computer Science 2026-02-12 Marc Lanctot , Kate Larson , Ian Gemp , Michael Kaisers

Agent-based editing models have substantially advanced interactive experiences, processing quality, and creative flexibility. However, two critical challenges persist: (1) instruction hallucination, text-only chain-of-thought (CoT)…

Computer Vision and Pattern Recognition · Computer Science 2025-12-05 Yunlong Lin , Linqing Wang , Kunjie Lin , Zixu Lin , Kaixiong Gong , Wenbo Li , Bin Lin , Zhenxi Li , Shiyi Zhang , Yuyang Peng , Wenxun Dai , Xinghao Ding , Chunyu Wang , Qinglin Lu

Even though many machine algorithms have been proposed for entity resolution, it remains very challenging to find a solution with quality guarantees. In this paper, we propose a novel HUman and Machine cOoperation (HUMO) framework for…

Databases · Computer Science 2018-04-03 Zhaoqiang Chen , Qun Chen , Fengfeng Fan , Yanyan Wang , Zhuo Wang , Youcef Nafa , Zhanhuai Li , Hailong Liu , Wei Pan

Large Language Models (LLMs) have become integral components in various autonomous agent systems. In this study, we present an exploration-based trajectory optimization approach, referred to as ETO. This learning method is designed to…

Computation and Language · Computer Science 2024-07-11 Yifan Song , Da Yin , Xiang Yue , Jie Huang , Sujian Li , Bill Yuchen Lin

Optimization remains a fundamental pillar of machine learning, yet existing methods often struggle to maintain stability and adaptability in dynamic, non linear systems, especially under uncertainty. We introduce AERO (Adversarial…

Machine Learning · Computer Science 2025-06-04 Karthikeyan Vaiapury

Deep research has revolutionized data analysis, yet data scientists still devote substantial time to manually crafting visualizations, highlighting the need for robust automation from natural language queries. However, current systems…

Artificial Intelligence · Computer Science 2025-10-06 Zichen Chen , Jiefeng Chen , Sercan Ö. Arik , Misha Sra , Tomas Pfister , Jinsung Yoon

The rise of AI agents is transforming how software can be built. The promise of agents is that developers might write code quicker, delegate multiple tasks to different agents, and even write a full piece of software purely out of natural…

Software Engineering · Computer Science 2025-12-17 Ruanqianqian Huang , Avery Reyna , Sorin Lerner , Haijun Xia , Brian Hempel

Evaluating instruction-guided image edits requires rewards that reflect subtle human preferences, yet current reward models typically depend on large-scale preference annotation and additional model training. This creates a data-efficiency…