English
Related papers

Related papers: PIRA-Bench: A Transition from Reactive GUI Agents …

200 papers

Agents, as user-centric tools, are increasingly deployed for human task delegation, assisting with a broad spectrum of requests by generating thoughts, engaging with user proxies, and producing action plans. However, agents based on large…

Multiagent Systems · Computer Science 2024-10-02 Wenyue Hua , Mengting Wan , Shashank Vadrevu , Ryan Nadel , Yongfeng Zhang , Chi Wang

The technical foundations of recommender systems have progressed from collaborative filtering to complex neural models and, more recently, large language models. Despite these technological advances, deployed systems often underserve their…

Information Retrieval · Computer Science 2026-03-10 Kesha Ou , Chenghao Wu , Xiaolei Wang , Bowen Zheng , Wayne Xin Zhao , Weitao Li , Long Zhang , Sheng Chen , Ji-Rong Wen

The emergence of generative AI (GenAI) models, including large language models and text-to-image models, has significantly advanced the synergy between humans and AI with not only their outstanding capability but more importantly, the…

Human-Computer Interaction · Computer Science 2025-03-05 Leixian Shen , Haotian Li , Yifang Wang , Xing Xie , Huamin Qu

Current personalized recommender systems predominantly rely on static offline data for algorithm design and evaluation, significantly limiting their ability to capture long-term user preference evolution and social influence dynamics in…

Multiagent Systems · Computer Science 2025-05-28 Hailin Zhong , Hanlin Wang , Yujun Ye , Meiyi Zhang , Shengxin Zhu

Reasoning models have recently shown remarkable progress in domains such as math and coding. However, their expert-level abilities in math and coding contrast sharply with their performance in long-horizon, interactive tasks such as web…

Computation and Language · Computer Science 2025-10-13 Xiao Yu , Baolin Peng , Michel Galley , Hao Cheng , Qianhui Wu , Janardhan Kulkarni , Suman Nath , Zhou Yu , Jianfeng Gao

Dashboards are powerful visualization tools for data-driven decision-making, integrating multiple interactive views that allow users to explore, filter, and navigate data. Unlike static charts, dashboards support rich interactivity, which…

The vision of a broadly capable and goal-directed agent, such as an Internet-browsing agent in the digital world and a household humanoid in the physical world, has rapidly advanced, thanks to the generalization capability of foundation…

Machine Learning · Computer Science 2024-12-18 Yifei Zhou , Qianlan Yang , Kaixiang Lin , Min Bai , Xiong Zhou , Yu-Xiong Wang , Sergey Levine , Erran Li

Multi-agent reinforcement learning faces fundamental challenges that conventional approaches have failed to overcome: exponentially growing joint action spaces, non-stationary environments where simultaneous learning creates moving targets,…

Artificial Intelligence · Computer Science 2025-07-15 Hang Wang , Junshan Zhang

Computer use agents (CUA) are systems that automatically interact with graphical user interfaces (GUIs) to complete tasks. CUA have made significant progress with the advent of large vision-language models (VLMs). However, these agents…

Artificial Intelligence · Computer Science 2025-06-04 Man Luo , David Cobbley , Xin Su , Shachar Rosenman , Vasudev Lal , Shao-Yen Tseng , Phillip Howard

Large Language Model (LLM)-driven conversational search is shifting information retrieval from reactive keyword matching to proactive, open-ended dialogues. In this context, Conversation Starters are widely deployed to provide personalized…

Information Retrieval · Computer Science 2026-05-08 Yiqing Wu , Haoming Li , Guanyu Jiang , Jiahao Liang , Yongchun Zhu , Jingwu Chen , Feng Zhang

With the rapid advancement of generative artificial intelligence technology, Graphical User Interface (GUI) agents have demonstrated tremendous potential for autonomously managing daily tasks through natural language instructions. However,…

Computer Vision and Pattern Recognition · Computer Science 2025-08-14 Fengxian Ji , Jingpu Yang , Zirui Song , Yuanxi Wang , Zhexuan Cui , Yuke Li , Qian Jiang , Miao Fang , Xiuying Chen

We introduce Interactionalism as a new set of guiding principles and heuristics for the design and architecture of learning now available due to Generative AI (GenAI) platforms. Specifically, we articulate interactional intelligence as a…

Human-Computer Interaction · Computer Science 2025-01-03 Mihnea C. Moldoveanu , George Siemens

Recent advancements in Large Language Models (LLMs) have catalyzed a paradigm shift from static prediction systems to agentic AI agents capable of reasoning, interacting with tools, and adapting to complex tasks. While LLM-based agentic…

Computer Vision and Pattern Recognition · Computer Science 2025-07-24 Nima Fathi , Amar Kumar , Tal Arbel

Measuring GUI task difficulty is crucial for user behavior analysis and agent capability evaluation. Yet, existing benchmarks typically quantify difficulty based on motor actions (e.g., step counts), overlooking the cognitive demands…

Human-Computer Interaction · Computer Science 2025-11-13 Yiwen Yin , Zhian Hu , Xiaoxi Xu , Chun Yu , Xintong Wu , Wenyu Fan , Yuanchun Shi

Building agents with adaptive behavior in cooperative tasks stands as a paramount goal in the realm of multi-agent systems. Current approaches to developing cooperative agents rely primarily on learning-based methods, whose policy…

Recent advances in AI-assisted programming have empowered agents to execute complex workflows via command-line interfaces, however, existing benchmarks are limited by short task horizons, data contamination from GitHub scraping, and a lack…

Graphical User Interface (GUI) action grounding is a critical step in GUI automation that maps language instructions to actionable elements on GUI screens. Most recent works of GUI action grounding leverage large GUI datasets to fine-tune…

Computation and Language · Computer Science 2025-01-28 Yue Fan , Handong Zhao , Ruiyi Zhang , Yu Shen , Xin Eric Wang , Gang Wu

Modeling multi-agent systems requires understanding how agents interact. Such systems are often difficult to model because they can involve a variety of types of interactions that layer together to drive rich social behavioral dynamics.…

Machine Learning · Computer Science 2023-01-26 Fan-Yun Sun , Isaac Kauvar , Ruohan Zhang , Jiachen Li , Mykel Kochenderfer , Jiajun Wu , Nick Haber

Despite significant advancements in Large Language Models (LLMs) and Large Vision-Language Models (LVLMs), current models still face substantial challenges in handling complex, multi-turn, and visually-grounded tasks that demand deep…

Computation and Language · Computer Science 2025-08-22 Seungmin Han , Haeun Kwon , Ji-jun Park , Taeyang Yoon

Large language models (LLMs) have advanced conversational AI assistants. However, systematically evaluating how well these assistants apply personalization--adapting to individual user preferences while completing tasks--remains…

Computation and Language · Computer Science 2025-06-12 Zheng Zhao , Clara Vania , Subhradeep Kayal , Naila Khan , Shay B. Cohen , Emine Yilmaz