English
Related papers

Related papers: UI-Genie: A Self-Improving Approach for Iterativel…

200 papers

Existing online benchmarks for mobile GUI agents remain largely app-centric and task-homogeneous, failing to reflect the diversity and instability of real-world mobile usage. To this end, we introduce VenusBench-Mobile, a challenging online…

Human-Computer Interaction · Computer Science 2026-04-09 Yichen Gong , Zhuohan Cai , Sunhao Dai , Yuqi Zhou , Zhangxuan Gu , Changhua Meng , Shuheng Shen

With the rapid development of Large Vision Language Models, the focus of Graphical User Interface (GUI) agent tasks shifts from single-screen tasks to complex screen navigation challenges. However, real-world GUI environments, such as PC…

Computer Vision and Pattern Recognition · Computer Science 2025-12-03 Haolong Yan , Yeqing Shen , Xin Huang , Jia Wang , Kaijun Tan , Zhixuan Liang , Hongxin Li , Zheng Ge , Osamu Yoshie , Si Li , Xiangyu Zhang , Daxin Jiang

Graphical user interface (GUI) agents have advanced rapidly but still struggle with complex tasks involving novel UI elements, long-horizon actions, and personalized trajectories. In this work, we introduce Instruction Agent, a GUI agent…

Artificial Intelligence · Computer Science 2025-09-10 Yinheng Li , Hailey Hultquist , Justin Wagle , Kazuhito Koishida

Generative Reward Models (GRMs) provide greater flexibility than scalar reward models in capturing human preferences, but their effectiveness is limited by poor reasoning capabilities. This often results in incomplete or overly speculative…

Computation and Language · Computer Science 2025-06-23 Bin Chen , Xinzge Gao , Chuanrui Hu , Penghang Yu , Hua Zhang , Bing-Kun Bao

The rapid development of large vision-language model (VLM) has greatly promoted the research of GUI agent. However, GUI agents still face significant challenges in handling long-horizon tasks. First, single-agent models struggle to balance…

Artificial Intelligence · Computer Science 2026-03-05 Zehao Deng , Tianjie Ju , Zheng Wu , Zhuosheng Zhang , Gongshen Liu

Scaling generalist GUI agents is hindered by the data scalability bottleneck of expensive human demonstrations and the "distillation ceiling" of synthetic teacher supervision. To transcend these limitations, we propose UI-Oceanus, a…

The development of GUI agents could revolutionize the next generation of human-computer interaction. Motivated by this vision, we present MAI-UI, a family of foundation GUI agents spanning the full spectrum of sizes, including 2B, 8B, 32B,…

Computer Vision and Pattern Recognition · Computer Science 2025-12-29 Hanzhang Zhou , Xu Zhang , Panrong Tong , Jianan Zhang , Liangyu Chen , Quyu Kong , Chenglin Cai , Chen Liu , Yue Wang , Jingren Zhou , Steven Hoi

In classical Reinforcement Learning from Human Feedback (RLHF), Reward Models (RMs) serve as the fundamental signal provider for model alignment. As Large Language Models evolve into agentic systems capable of autonomous tool invocation and…

Artificial Intelligence · Computer Science 2026-05-12 Jiaxuan Wang , Yulan Hu , Wenjin Yang , Zheng Pan , Xin Li , Lan-Zhe Guo

Building autonomous agents that perceive and operate graphical user interfaces (GUIs) like humans has long been a vision in the field of artificial intelligence. Central to these agents is the capability for GUI interaction, which involves…

Computer Vision and Pattern Recognition · Computer Science 2025-09-23 Hongxin Li , Jingran Su , Jingfan Chen , Zheng Ju , Yuntao Chen , Qing Li , Zhaoxiang Zhang

We study how to endow GUI agents with scalable memory that help generalize across unfamiliar interfaces and long-horizon tasks. Prior GUI agents compress past trajectories into text tokens, which balloons context length and misses decisive…

Artificial Intelligence · Computer Science 2025-10-13 Wenyi Wu , Kun Zhou , Ruoxin Yuan , Vivian Yu , Stephen Wang , Zhiting Hu , Biwei Huang

This paper introduces the design and development of a framework that integrates a large language model (LLM) with a retrieval-augmented generation (RAG) approach leveraging both a knowledge graph and user interaction history. The framework…

Programming Languages · Computer Science 2026-03-27 Lalita Na Nongkhai , Jingyun Wang , Adam Wynn , Takahiko Mendori

AI research agents accelerate ML research by automating hypothesis generation, experimentation, and empirical refinement. Existing agent strategies range from greedy hill-climbing to tree search and evolutionary optimization, yet which…

With the rapid advancement of Vision-Language Models (VLMs), GUI-based mobile agents have emerged as a key development direction for intelligent mobile systems. However, existing agent models continue to face significant challenges in…

Multiagent Systems · Computer Science 2025-09-03 Cheng Zhang , Erhu Feng , Xi Zhao , Yisheng Zhao , Wangbo Gong , Jiahui Sun , Dong Du , Zhichao Hua , Yubin Xia , Haibo Chen

In recent years, significant progress has been made in both image generation and generated image detection. Despite their rapid, yet largely independent, development, these two fields have evolved distinct architectural paradigms: the…

Computer Vision and Pattern Recognition · Computer Science 2026-04-24 Yanran Zhang , Wenzhao Zheng , Yifei Li , Bingyao Yu , Yu Zheng , Lei Chen , Jiwen Lu , Jie Zhou

The lack of high-quality data for content-grounded generation tasks has been identified as a major obstacle to advancing these tasks. To address this gap, we propose Genie, a novel method for automatically generating high-quality…

Computation and Language · Computer Science 2024-01-26 Asaf Yehudai , Boaz Carmeli , Yosi Mass , Ofir Arviv , Nathaniel Mills , Assaf Toledo , Eyal Shnarch , Leshem Choshen

MLLM-based GUI agents have demonstrated strong capabilities in complex user interface interaction tasks. However, long-horizon scenarios remain challenging, as these agents are burdened with tasks beyond their intrinsic capabilities,…

Machine Learning · Computer Science 2026-04-16 Zhengxi Lu , Fei Tang , Guangyi Liu , Kaitao Song , Xu Tan , Jin Ma , Wenqi Zhang , Weiming Lu , Jun Xiao , Yueting Zhuang , Yongliang Shen

We introduce GAIN (Goal-Aligned Decision-Making under Imperfect Norms), a benchmark designed to evaluate how large language models (LLMs) balance adherence to norms against business goals. Existing benchmarks typically focus on abstract…

Computation and Language · Computer Science 2026-03-20 Masayuki Kawarada , Kodai Watanabe , Soichiro Murakami

Designing reward functions for efficiently guiding reinforcement learning (RL) agents toward specific behaviors is a complex task. This is challenging since it requires the identification of reward structures that are not sparse and that…

Machine Learning · Computer Science 2023-11-01 Dhawal Gupta , Yash Chandak , Scott M. Jordan , Philip S. Thomas , Bruno Castro da Silva

This study aims to develop an adaptive learning platform that leverages generative AI to automate assessment creation and feedback delivery. The platform provides self-correcting tests and personalised feedback that adapts to each learners…

Human-Computer Interaction · Computer Science 2025-11-07 Riya Gill , Ievgeniia Kuzminykh , Maher Salem , Bogdan Ghita

Automatic evaluation is crucial yet challenging for open-ended natural language generation, especially when rule-based metrics are infeasible. Compared with traditional methods, the recent LLM-as-a-Judge paradigms enable better and more…

Computation and Language · Computer Science 2026-02-03 Xinyu Hu , Yancheng He , Weixun Wang , Tao Feng , Li Lin , Jiashun Liu , Wenbo Su , Bo Zheng , Xiaojun Wan
‹ Prev 1 8 9 10 Next ›