中文
相关论文

相关论文: Test-Time Reinforcement Learning for GUI Grounding…

200 篇论文

Recent advancements highlight the importance of GRPO-based reinforcement learning methods and benchmarking in enhancing text-to-image (T2I) generation. However, current methods using pointwise reward models (RM) for scoring generated images…

计算机视觉与模式识别 · 计算机科学 2026-04-21 Yibin Wang , Zhimin Li , Yuhang Zang , Yujie Zhou , Jiazi Bu , Chunyu Wang , Qinglin Lu , Cheng Jin , Jiaqi Wang

Reinforcement Learning (RL) methods, exemplified by Group Relative Policy Optimization (GRPO) and its variants, play a central role in developing reasoning models. However, these methods often suffer from a critical overconfidence issue,…

机器学习 · 计算机科学 2025-12-24 Haotian Liu , Shuo Wang , Hongteng Xu

Pure-vision GUI agents provide universal interaction capabilities but suffer from severe efficiency bottlenecks due to the massive spatiotemporal redundancy inherent in high-resolution screenshots and historical trajectories. We identify…

计算机视觉与模式识别 · 计算机科学 2026-02-27 Zhou Xu , Bowen Zhou , Qi Wang , Shuwen Feng , Jingyu Xiao

Post-training GUI agents in interactive environments is critical for developing generalization and long-horizon planning capabilities. However, training on real-world applications is hindered by high latency, poor reproducibility, and…

人工智能 · 计算机科学 2026-02-17 Yuan Cao , Dezhi Ran , Mengzhou Wu , Yuzhe Guo , Xin Chen , Ang Li , Gang Cao , Gong Zhi , Hao Yu , Linyi Li , Wei Yang , Tao Xie

Mobile GUI agents excel at immediate reactive control but frequently fail in realistic, long-horizon tasks that require memory. This failure stems from a fundamental conflict between limited context windows and token-heavy screenshots. To…

计算与语言 · 计算机科学 2026-05-29 Junyang Wang , Haiyang Xu , Xi Zhang , Zhaoqing Zhu , Ming Yan , Jieping Ye , Jitao Sang

While Multimodal Large Language Models (MLLMs) have advanced GUI navigation agents, current approaches face limitations in cross-domain generalization and effective history utilization. We present a reasoning-enhanced framework that…

人工智能 · 计算机科学 2025-11-03 Tao Liu , Chongyu Wang , Rongjie Li , Yingchen Yu , Xuming He , Bai Song

Ensuring safety in robotic systems remains a fundamental challenge, especially when deploying offline policy-learning methods such as imitation learning in dynamic environments. Traditional behavior cloning (BC) often fails to generalize…

机器人学 · 计算机科学 2025-09-30 Mumuksh Tayal , Manan Tayal , Ravi Prakash

We study reinforcement learning (RL) in the setting of continuous time and space, for an infinite horizon with a discounted objective and the underlying dynamics driven by a stochastic differential equation. Built upon recent advances in…

机器学习 · 计算机科学 2023-10-19 Hanyang Zhao , Wenpin Tang , David D. Yao

Recent advances in reinforcement learning (RL) have delivered strong reasoning capabilities in natural image domains, yet their potential for Earth Observation (EO) remains largely unexplored. EO tasks introduce unique challenges, spanning…

计算机视觉与模式识别 · 计算机科学 2025-10-15 Mustansar Fiaz , Hiyam Debary , Paolo Fraccaro , Danda Paudel , Luc Van Gool , Fahad Khan , Salman Khan

Reinforcement learning (RL) is attracting increasing interests in autonomous driving due to its potential to solve complex classification and control problems. However, existing RL algorithms are rarely applied to real vehicles for two…

机器学习 · 计算机科学 2020-03-04 Lu Wen , Jingliang Duan , Shengbo Eben Li , Shaobing Xu , Huei Peng

Large Language Models (LLMs) are increasingly aligned with human preferences through Reinforcement Learning from Human Feedback (RLHF). Among RLHF methods, Group Relative Policy Optimization (GRPO) has gained attention for its simplicity…

计算与语言 · 计算机科学 2025-09-25 Yuhang Zhou , Jing Zhu , Shengyi Qian , Zhuokai Zhao , Xiyao Wang , Xiaoyu Liu , Ming Li , Paiheng Xu , Wei Ai , Furong Huang

Conventional visual object trackers localize targets using handcrafted spatial priors, often in the form of heatmaps. Such priors provide only surrogate supervision and are poorly aligned with tracking optimization and evaluation metrics,…

计算机视觉与模式识别 · 计算机科学 2026-05-20 Xin Chen , Chuanyu Sun , Jiao Xu , Houwen Peng , Dong Wang , Huchuan Lu , Kede Ma

Reinforcement learning with verifiable rewards (RLVR) has driven breakthroughs in domains such as math, tool-use, and software engineering, yet its extension to computer-use agents (CUAs) has been bottlenecked by the scarcity of scalable…

Existing studies on constrained reinforcement learning (RL) may obtain a well-performing policy in the training environment. However, when deployed in a real environment, it may easily violate constraints that were originally satisfied…

机器学习 · 计算机科学 2024-05-06 Zhongchang Sun , Sihong He , Fei Miao , Shaofeng Zou

GUI Process Automation (GPA) is a lightweight but general vision-based Robotic Process Automation (RPA), which enables fast and stable process replay with only a single demo. Addressing the fragility of traditional RPA and the…

计算机视觉与模式识别 · 计算机科学 2026-04-07 Zirui Zhao , Jun Hao Liew , Yan Yang , Wenzhuo Yang , Ziyang Luo , Doyen Sahoo , Silvio Savarese , Junnan Li

Graphical User Interface (GUI) agents extend large language models from text generation to action execution in real-world digital environments. Unlike conversational systems, GUI agents perform irreversible operations such as submitting…

机器学习 · 计算机科学 2026-02-25 Yucheng Shi , Wenhao Yu , Jingyuan Huang , Wenlin Yao , Wenhu Chen , Ninghao Liu

Editing images using natural language instructions has become a natural and expressive way to modify visual content; yet, evaluating the performance of such models remains challenging. Existing evaluation approaches often rely on image-text…

计算机视觉与模式识别 · 计算机科学 2025-07-28 Yusu Qian , Jiasen Lu , Tsu-Jui Fu , Xinze Wang , Chen Chen , Yinfei Yang , Wenze Hu , Zhe Gan

Reasoning over dynamic visual content remains a central challenge for multimodal large language models. Recent thinking models generate explicit reasoning traces for interpretability; however, their reasoning often appears convincing while…

计算机视觉与模式识别 · 计算机科学 2025-12-09 Muhammad Maaz , Hanoona Rasheed , Fahad Shahbaz Khan , Salman Khan

Test-time training (TTT) adapts language models through gradient-based updates at inference. But is adaptation the right strategy? We study compute-optimal test-time strategies for verifiable execution-grounded (VEG) tasks, domains like GPU…

机器学习 · 计算机科学 2026-02-10 Jarrod Barnes

Recent advances in reinforcement learning (RL) have significantly improved the complex reasoning capabilities of large language models (LLMs). Despite these successes, existing methods mainly focus on single-domain RL (e.g., mathematics)…

人工智能 · 计算机科学 2025-11-20 Baolong Bi , Shenghua Liu , Yiwei Wang , Siqian Tong , Lingrui Mei , Yuyao Ge , Yilong Xu , Jiafeng Guo , Xueqi Cheng