English
Related papers

Related papers: Enhancing Visual Grounding for GUI Agents via Self…

200 papers

Grounding language in perception and action is a key challenge when building situated agents that can interact with humans, or other agents, via language. In the past, addressing this challenge has required manually designing the language…

Machine Learning · Computer Science 2025-10-28 Andrew C. Li , Toryn Q. Klassen , Andrew Wang , Parand A. Alamdari , Sheila A. McIlraith

Simulation offers a scalable and low-cost way to enrich vision-language-action (VLA) training, reducing reliance on expensive real-robot demonstrations. However, most sim-real co-training methods rely on supervised fine-tuning (SFT), which…

Robotics · Computer Science 2026-03-09 Liangzhi Shi , Shuaihang Chen , Feng Gao , Yinuo Chen , Kang Chen , Tonghe Zhang , Hongzhi Zang , Weinan Zhang , Chao Yu , Yu Wang

GUI grounding is a critical capability for enabling GUI agents to execute tasks such as clicking and dragging. However, in complex scenarios like the ScreenSpot-Pro benchmark, existing models often suffer from suboptimal performance.…

Computer Vision and Pattern Recognition · Computer Science 2026-05-08 Borui Zhang , Bo Zhang , Bo Wang , Wenzhao Zheng , Yuhao Cheng , Liang Tang , Yiqiang Yan , Jie Zhou , Jiwen Lu

Recent advances in video world modeling have enabled large-scale generative models to simulate embodied environments with high visual fidelity, providing strong priors for prediction, planning, and control. Yet, despite their realism, these…

Computer Vision and Pattern Recognition · Computer Science 2026-02-10 Haoyang He , Jay Patrikar , Dong-Ki Kim , Max Smith , Daniel McGann , Ali-akbar Agha-mohammadi , Shayegan Omidshafiei , Sebastian Scherer

Graphical User Interface (GUI) agents show amazing abilities in assisting human-computer interaction, automating human user's navigation on digital devices. An ideal GUI agent is expected to achieve high accuracy, low latency, and…

Computer Vision and Pattern Recognition · Computer Science 2025-04-17 Zhiyuan Huang , Ziming Cheng , Junting Pan , Zhaohui Hou , Mingjie Zhan

Reinforcement Learning (RL) is essential for evolving Large Language Models (LLMs) into autonomous agents capable of long-horizon planning, yet a practical recipe for scaling RL in complex, multi-turn environments remains elusive. This…

Machine Learning · Computer Science 2026-03-24 Xixi Wu , Qianguo Sun , Ruiyang Zhang , Chao Song , Junlong Wu , Yiyan Qi , Hong Cheng

In this paper, we explore the utilization of natural language to drive transfer for reinforcement learning (RL). Despite the wide-spread application of deep RL techniques, learning generalized policy representations that work across domains…

Computation and Language · Computer Science 2018-12-07 Karthik Narasimhan , Regina Barzilay , Tommi Jaakkola

The remarkable progress of reinforcement learning (RL) is intrinsically tied to the environments used to train and evaluate artificial agents. Moving beyond traditional qualitative reviews, this work presents a large-scale, data-driven…

Artificial Intelligence · Computer Science 2026-04-14 Lijing Luo , Yiben Luo , Alexey Gorbatovski , Sergey Kovalchuk , Xiaodan Liang

Underwater target localization using range-only and single-beacon (ROSB) techniques with autonomous vehicles has been used recently to improve the limitations of more complex methods, such as long baseline and ultra-short baseline systems.…

Robotics · Computer Science 2023-01-18 Ivan Masmitja , Mario Martin , Kakani Katija , Spartacus Gomariz , Joan Navarro

Most deep reinforcement learning (RL) algorithms distill experience into parametric behavior policies or value functions via gradient updates. While effective, this approach has several disadvantages: (1) it is computationally expensive,…

Vision-language model (VLM) fine-tuning for application-specific visual grounding based on natural language instructions has become one of the most popular approaches for learning-enabled autonomous systems. However, such fine-tuning relies…

Computer Vision and Pattern Recognition · Computer Science 2025-02-03 Joshua R. Waite , Md. Zahid Hasan , Qisai Liu , Zhanhong Jiang , Chinmay Hegde , Soumik Sarkar

Agent decision making using Reinforcement Learning (RL) heavily relies on either a model or simulator of the environment (e.g., moving in an 8x8 maze with three rooms, playing Chess on an 8x8 board). Due to this dependence, small changes in…

Artificial Intelligence · Computer Science 2023-09-20 Wenjun Li , Pradeep Varakantham , Dexun Li

Graphical User Interface (GUI) agents powered by Large Vision-Language Models (LVLMs) have emerged as a revolutionary approach to automating human-machine interactions, capable of autonomously operating personal devices (e.g., mobile…

Computation and Language · Computer Science 2025-09-25 Ziang Ye , Yang Zhang , Wentao Shi , Xiaoyu You , Fuli Feng , Tat-Seng Chua

Reinforcement learning (RL) provides a naturalistic framing for learning through trial and error, which is appealing both because of its simplicity and effectiveness and because of its resemblance to how humans and animals acquire skills…

Machine Learning · Computer Science 2022-08-09 Archit Sharma , Kelvin Xu , Nikhil Sardana , Abhishek Gupta , Karol Hausman , Sergey Levine , Chelsea Finn

Table reasoning, encompassing tasks such as table question answering, fact verification, and text-to-SQL, requires precise understanding of structured tabular data, coupled with numerical computation and code manipulation for effective…

Computation and Language · Computer Science 2025-06-03 Fangyu Lei , Jinxiang Meng , Yiming Huang , Tinghong Chen , Yun Zhang , Shizhu He , Jun Zhao , Kang Liu

Vision-language agents that orchestrate specialized tools for image restoration (IR) have emerged as a promising method, yet most existing frameworks operate in a training-free manner. They rely on heuristic task scheduling and exhaustive…

Computer Vision and Pattern Recognition · Computer Science 2026-03-31 Yisheng Zhang , Guoli Jia , Haote Hu , Shanxu Zhao , Kaikai Zhao , Long Sun , Xinwei Long , Kai Tian , Che Jiang , Zhaoxiang Liu , Kai Wang , Shiguo Lian , Kaiyan Zhang , Bowen Zhou

Reinforcement learning (RL) is a powerful way to adapt foundation models to specialized tasks, but its reliance on large-scale human-labeled data limits broad adoption. We introduce Synthetic Data RL, a simple and general framework that…

Computation and Language · Computer Science 2025-05-26 Yiduo Guo , Zhen Guo , Chuanwei Huang , Zi-Ang Wang , Zekai Zhang , Haofei Yu , Huishuai Zhang , Yikang Shen

Building speech deepfake detection models that are generalizable to unseen attacks remains a challenging problem. Although the field has shifted toward a pre-training and fine-tuning paradigm using speech foundation models, most approaches…

Audio and Speech Processing · Electrical Eng. & Systems 2026-03-04 Xin Wang , Ge Wanying , Junichi Yamagishi

Reinforcement Learning (RL), a subfield of Artificial Intelligence (AI), focuses on training agents to make decisions by interacting with their environment to maximize cumulative rewards. This paper provides an overview of RL, covering its…

Artificial Intelligence · Computer Science 2024-12-04 Majid Ghasemi , Dariush Ebrahimi

In recent years, unmanned aerial vehicle (UAV) related technology has expanded knowledge in the area, bringing to light new problems and challenges that require solutions. Furthermore, because the technology allows processes usually carried…

Artificial Intelligence · Computer Science 2022-12-08 Cristian Millán-Arias , Ruben Contreras , Francisco Cruz , Bruno Fernandes
‹ Prev 1 8 9 10 Next ›