English
Related papers

Related papers: ANCHOR: Branch-Point Data Generation for GUI Agent…

200 papers

Graphical user interfaces (GUI) automation agents are emerging as powerful tools, enabling humans to accomplish increasingly complex tasks on smart devices. However, users often inadvertently omit key information when conveying tasks, which…

Computer Vision and Pattern Recognition · Computer Science 2025-08-26 Ziming Cheng , Zhiyuan Huang , Junting Pan , Zhaohui Hou , Mingjie Zhan

This paper develops a control-theoretic framework for analyzing agentic systems embedded within feedback control loops, where an AI agent may adapt controller parameters, select among control strategies, invoke external tools, reconfigure…

Systems and Control · Electrical Eng. & Systems 2026-03-26 Ali Eslami , Jiangbo Yu

Agentic workflows in large language model systems integrate retrieval, reasoning, and memory, but existing frameworks suffer from scalability and reproducibility limitations due to fragmented data orchestration, serialization overhead, and…

Distributed, Parallel, and Cluster Computing · Computer Science 2026-05-05 Arup Kumar Sarker , Mills Staylor , Aymen Alsaadi , Gregor von Laszewski , Shantenu Jha , Geoffrey Fox

Reinforcement learning has empowered large language models to act as intelligent agents, yet training them for long-horizon tasks remains challenging due to the scarcity of high-quality trajectories, especially under limited resources.…

Machine Learning · Computer Science 2026-01-29 Jinyang Wu , Shuo Yang , Changpeng Yang , Yuhao Shen , Shuai Zhang , Zhengqi Wen , Jianhua Tao

Graphical User Interface (GUI) agents have made significant progress in automating digital tasks through the utilization of computer vision and language models. Nevertheless, existing agent systems encounter notable limitations. Firstly,…

Artificial Intelligence · Computer Science 2025-06-24 Jinjie Wei , Jiyao Liu , Lihao Liu , Ming Hu , Junzhi Ning , Mingcheng Li , Weijie Yin , Junjun He , Xiao Liang , Chao Feng , Dingkang Yang

Graphical User Interface (GUI) task automation constitutes a critical frontier in artificial intelligence research. While effective GUI agents synergistically integrate planning and grounding capabilities, current methodologies exhibit two…

Artificial Intelligence · Computer Science 2025-11-17 Yuan Zhao , Hualei Zhu , Tingyu Jiang , Shen Li , Xiaohang Xu , Hao Henry Wang

Despite the rapid progress of multimodal large language models in building Graphical User Interface (GUI) agents, their real-world task completion is fundamentally bottlenecked by a lack of world knowledge about GUI operations. Existing…

Computation and Language · Computer Science 2026-05-28 Zheng Wu , Chengcheng Han , Zhengxi Lu , Tianjie Ju , Yanyu Chen , Qi Gu , Xunliang Cai , Zhuosheng Zhang

We study parallel test-time scaling for long-horizon agentic tasks such as agentic search and deep research, where multiple rollouts are generated in parallel and aggregated into a final response. While such scaling has proven effective for…

Computation and Language · Computer Science 2026-04-14 Yoonsang Lee , Howard Yen , Xi Ye , Danqi Chen

GUI agents are rapidly becoming a new interaction to software, allowing people to navigate web, desktop and mobile rather than execute them click by click. Yet ``agent'' is described with radically different degrees of autonomy, obscuring…

Software Engineering · Computer Science 2026-02-13 Sidong Feng , Chunyang Chen

Advanced agentic intelligence is a prerequisite for deploying Large Language Models in practical, real-world applications. Diverse real-world APIs demand precise, robust function-calling intelligence, which needs agents to develop these…

We present Seed1.8, a foundation model aimed at generalized real-world agency: going beyond single-turn prediction to multi-turn interaction, tool use, and multi-step execution. Seed1.8 keeps strong LLM and vision-language performance while…

Artificial Intelligence · Computer Science 2026-04-20 Bytedance Seed

As personal agents evolve to handle complex, user-centric tasks, static plain-text chat is rapidly becoming a bottleneck. Generative UI emerges as the necessary new interface layer, dynamically synthesizing the right controls, options, and…

Human-Computer Interaction · Computer Science 2026-05-26 Fancy Kong , Congjie Zheng , Murphy Zhuang , Rio Yang , Sueky Zhang , Hao Fu , Gene Jin , Song Cao , Kaijie Chen , Andrew Chen , Pony Ma

Recent advances in Multimodal Large Language Models (MLLMs) have substantially driven the progress of autonomous agents for Graphical User Interface (GUI). Nevertheless, in real-world applications, GUI agents are often faced with…

Artificial Intelligence · Computer Science 2026-02-17 Yibo Wang , Guangda Huzhang , Yuwei Hu , Yu Xia , Shiyin Lu , Qing-Guo Chen , Zhao Xu , Weihua Luo , Kaifu Zhang , Lijun Zhang

Automation of existing Graphical User Interfaces (GUIs) is important but hard to achieve. Upstream of making the GUI user-accessible or somehow scriptable, even the data-collection to understand the original interface poses significant…

Human-Computer Interaction · Computer Science 2025-04-15 Iason Chaimalas , Arnas Vyšniauskas , Gabriel Brostow

Current evaluations of agents remain centered around one-shot task completion, failing to account for the inherently iterative and collaborative nature of many real-world problems, where human goals are often underspecified and evolve. We…

Long-horizon embodied agents increasingly delegate navigation, search, approach, and manipulation to specialist executors. As these executors become stronger, the main bottleneck shifts from local skill execution to maintaining a coherent…

Robotics · Computer Science 2026-05-20 Shuhan Guo , Kun Zhang , Haifei Liu , Xingyu Gao , Yongqi Zhang , Yaqing Wang , Quanming Yao

Software testing has progressed toward intelligent automation, yet current AI-based test generators still suffer from static, single-shot outputs that frequently produce invalid, redundant, or non-executable tests due to the lack of…

Software Engineering · Computer Science 2026-01-07 Saba Naqvi , Mohammad Baqar , Nawaz Ali Mohammad

Recent advancements in vision-language models have increased interest in Device-Control Agents (DC agents) for managing graphical user interfaces (GUIs). With the growing complexity and integration of such agents into various applications,…

Artificial Intelligence · Computer Science 2025-09-29 Shaoqing Zhang , Kehai Chen , Zhuosheng Zhang , Rumei Li , Rongxiang Weng , Yang Xiang , Min Zhang

Real-world data is represented in both structured (e.g., graph connections) and unstructured (e.g., textual, visual information) formats, encompassing complex relationships that include explicit links (such as social connections and user…

Artificial Intelligence · Computer Science 2024-12-24 Yuhao Yang , Jiabin Tang , Lianghao Xia , Xingchen Zou , Yuxuan Liang , Chao Huang

Machine learning on graphs has made substantial progress across domains such as molecular property prediction and chip design. Yet benchmarking practices remain fragmented, often relying on narrow, task-specific datasets and inconsistent…