English
Related papers

Related papers: Agentic Reward Modeling: Verifying GUI Agent via O…

200 papers

Reinforcement Learning from Verifiable Rewards (RLVR) has been widely adopted as the de facto method for enhancing the reasoning capabilities of large language models and has demonstrated notable success in verifiable domains like math and…

Computation and Language · Computer Science 2025-06-24 Jeff Da , Clinton Wang , Xiang Deng , Yuntao Ma , Nikhil Barhate , Sean Hendryx

Verifiers have been demonstrated to enhance LLM reasoning via test-time scaling (TTS). Yet, they face significant challenges in complex domains. Error propagation from incorrect intermediate reasoning can lead to false positives for…

A key challenge in training Vision-Language Model (VLM) agents, compared to Language Model (LLM) agents, lies in the shift from textual states to complex visual observations. This transition introduces partial observability and demands…

Autonomous GUI agents based on vision-language models (VLMs) often assume deterministic environment responses, generating actions without verifying whether previous operations succeeded. In real-world settings with network latency,…

Computation and Language · Computer Science 2026-04-08 Yuzhe Zhang , Xianwei Xue , Xingyong Wu , Mengke Chen , Chen Liu , Xinran He , Run Shao , Feiran Liu , Huanmin Xu , Qiutong Pan , Haiwei Wang

GUI agents powered by vision-language models (VLMs) show promise in automating complex digital tasks. However, their effectiveness in real-world applications is often limited by scarce training data and the inherent complexity of these…

Computation and Language · Computer Science 2025-09-30 Ran Xu , Kaixin Ma , Wenhao Yu , Hongming Zhang , Joyce C. Ho , Carl Yang , Dong Yu

Training Vision-Language Models (VLMs) for Graphical User Interfaces (GUI) agents via Reinforcement Learning (RL) faces critical challenges: environment-based RL requires costly interactions, while environment-free methods struggle with…

Machine Learning · Computer Science 2025-02-27 Jiani Zheng , Lu Wang , Fangkai Yang , Chaoyun Zhang , Lingrui Mei , Wenjie Yin , Qingwei Lin , Dongmei Zhang , Saravan Rajmohan , Qi Zhang

Reinforcement learning from verifiable rewards (RLVR) has improved the reasoning abilities of large language models (LLMs), but most existing approaches rely on sparse outcome-level feedback. This sparsity creates a credit assignment…

Artificial Intelligence · Computer Science 2026-05-28 Huining Yuan , Zelai Xu , Huaijie Wang , Xiangmin Yi , Jiaxuan Gao , Xiao-Ping Zhang , Yu Wang , Chao Yu , Yi Wu

Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have demonstrated significant potential in single-turn reasoning tasks. With the paradigm shift toward self-evolving agentic learning, models are increasingly expected…

Artificial Intelligence · Computer Science 2026-04-21 Xinshun Feng , Xinhao Song , Lijun Li , Gongshen Liu , Jing Shao

Reward models (RMs) are crucial for the training and inference-time scaling up of large language models (LLMs). However, existing reward models primarily focus on human preferences, neglecting verifiable correctness signals which have shown…

Computation and Language · Computer Science 2025-02-27 Hao Peng , Yunjia Qi , Xiaozhi Wang , Zijun Yao , Bin Xu , Lei Hou , Juanzi Li

As reinforcement learning continues to scale the training of large language model-based agents, reliably verifying agent behaviors in complex environments has become increasingly challenging. Existing approaches rely on rule-based verifiers…

Artificial Intelligence · Computer Science 2026-04-21 Wentao Shi , Yu Wang , Yuyang Zhao , Yuxin Chen , Fuli Feng , Xueyuan Hao , Xi Su , Qi Gu , Hui Su , Xunliang Cai , Xiangnan He

Graphical User Interface (GUI) agents have emerged as a promising paradigm for intelligent systems that perceive and interact with graphical interfaces visually. Yet supervised fine-tuning alone cannot handle long-horizon credit assignment,…

Artificial Intelligence · Computer Science 2026-05-01 Junan Hu , Jian Liu , Jingxiang Lai , Jiarui Hu , Yiwei Sheng , Shuang Chen , Jian Li , Dazhao Du , Song Guo

Agentic reinforcement learning (RL) holds great promise for the development of autonomous agents under complex GUI tasks, but its scalability remains severely hampered by the verification of task completion. Existing task verification is…

Computation and Language · Computer Science 2026-01-07 Shaofei Cai , Yulei Qin , Haojia Lin , Zihan Xu , Gang Li , Yuchen Shi , Zongyi Li , Yong Mao , Siqi Cai , Xiaoyu Tan , Yitao Liang , Ke Li , Xing Sun

Large language models (LLMs) excel at logical and algorithmic reasoning, yet their emotional intelligence (EQ) still lags far behind their cognitive prowess. While reinforcement learning from verifiable rewards (RLVR) has advanced in other…

Recent advancements in visual language models (VLMs) have notably enhanced their capabilities in handling complex Graphical User Interface (GUI) interaction tasks. Despite these improvements, current frameworks often struggle to generate…

Computation and Language · Computer Science 2025-04-23 Zhiyuan Hu , Shiyun Xiong , Yifan Zhang , See-Kiong Ng , Anh Tuan Luu , Bo An , Shuicheng Yan , Bryan Hooi

Reward models are critical for aligning vision-language systems with human preferences, yet current approaches suffer from hallucination, weak visual grounding, and an inability to use tools for verification, limiting their reliability on…

Computer Vision and Pattern Recognition · Computer Science 2025-12-05 Shengyuan Ding , Xinyu Fang , Ziyu Liu , Yuhang Zang , Yuhang Cao , Xiangyu Zhao , Haodong Duan , Xiaoyi Dong , Jianze Liang , Bin Wang , Conghui He , Dahua Lin , Jiaqi Wang

Graphical User Interface (GUI) agents, driven by Multi-modal Large Language Models (MLLMs), have emerged as a promising paradigm for enabling intelligent interaction with digital systems. This paper provides a structured survey of recent…

Artificial Intelligence · Computer Science 2025-05-14 Jiahao Li , Kaer Huang

Designing reward functions for continuous-control robotics often leads to subtle misalignments or reward hacking, especially in complex tasks. Preference-based RL mitigates some of these pitfalls by learning rewards from comparative…

Artificial Intelligence · Computer Science 2025-03-19 Anukriti Singh , Amisha Bhaskar , Peihong Yu , Souradip Chakraborty , Ruthwik Dasyam , Amrit Bedi , Pratap Tokekar

Reward is critical to the evaluation and training of large language models (LLMs). However, existing rule-based or model-based reward methods struggle to generalize to GUI agents, where access to ground-truth trajectories or application…

Artificial Intelligence · Computer Science 2026-04-16 Gaole Dai , Shiqi Jiang , Ting Cao , Yuqing Yang , Yuanchun Li , Rui Tan , Mo Li , Lili Qiu

The development of autonomous agents for complex, long-horizon tasks is a central goal in AI. However, dominant training paradigms face a critical limitation: reinforcement learning (RL) methods that optimize solely for final task success…

Machine Learning · Computer Science 2025-07-31 Zijing Zhang , Ziyang Chen , Mingxiao Li , Zhaopeng Tu , Xiaolong Li

Reinforcement learning with verifiable rewards (RLVR) has become the mainstream technique for training LLM agents. However, RLVR highly depends on well-crafted task queries and corresponding ground-truth answers to provide accurate rewards,…

Machine Learning · Computer Science 2026-05-20 Hongliang Lu , Yuhang Wen , Pengyu Cheng , Ruijin Ding , Jiaqi Guo , Haotian Xu , Chutian Wang , Haonan Chen , Xiaoxi Jiang , Guanjun Jiang
‹ Prev 1 2 3 10 Next ›