English
Related papers

Related papers: EE-MCP: Self-Evolving MCP-GUI Agents via Automated…

200 papers

Effective multi-agent systems cannot be designed by selecting prompts or communication graphs in isolation. Agent behavior depends on the information an agent receives, while the usefulness of a communication edge depends on how the…

Artificial Intelligence · Computer Science 2026-05-28 Yi Ding , Zijie Xuan , Haowei Zhou , Zhenyu Ju , Xiaoxiao Dong , Jingwen Zhang , Xingyu Zhu , Leixin Sun , Haochi Zhang

Online Reinforcement Learning (RL) offers a promising paradigm for enhancing GUI agents through direct environment interaction. However, its effectiveness is severely hindered by inefficient credit assignment in long-horizon tasks and…

Computer Vision and Pattern Recognition · Computer Science 2026-02-06 Han Xiao , Guozhi Wang , Hao Wang , Shilong Liu , Yuxiang Chai , Yue Pan , Yufeng Zhou , Xiaoxin Chen , Yafei Wen , Hongsheng Li

Tool calling has emerged as a critical capability for AI agents. In contrast to conventional tool calling frameworks that rely on static, provider-specific tool definitions, the Model Context Protocol (MCP) offers a unified interface to…

Current LLM agents are proficient at calling isolated APIs but struggle with the "last mile" of commercial software automation. In real-world scenarios, tools are not independent; they are atomic, interdependent, and prone to environmental…

Artificial Intelligence · Computer Science 2026-05-21 Yuanyang Li , Xue Yang , Longyue Wang , Weihua Luo , Hongyang Chen

(M)LLM-powered computer use agents (CUA) are emerging as a transformative technique to automate human-computer interaction. However, existing CUA benchmarks predominantly target GUI agents, whose evaluation methods are susceptible to UI…

Autonomous Graphical User Interface (GUI) agents often struggle with multi-step tasks due to constrained context windows and static policies that fail to adapt to dynamic environments. To address these limitations, this work proposes the…

Machine Learning · Computer Science 2026-05-19 Shilong Jin , Lanjun Wang , Zhuosheng Zhang

With advances in decision-making and reasoning capabilities, multimodal agents show strong potential in computer application scenarios. Past evaluations have mainly assessed GUI interaction skills, while tool invocation abilities, such as…

Computer Vision and Pattern Recognition · Computer Science 2025-11-12 Hongrui Jia , Jitong Liao , Xi Zhang , Haiyang Xu , Tianbao Xie , Chaoya Jiang , Ming Yan , Si Liu , Wei Ye , Fei Huang

Graphical User Interface (GUI) task automation constitutes a critical frontier in artificial intelligence research. While effective GUI agents synergistically integrate planning and grounding capabilities, current methodologies exhibit two…

Artificial Intelligence · Computer Science 2025-11-17 Yuan Zhao , Hualei Zhu , Tingyu Jiang , Shen Li , Xiaohang Xu , Hao Henry Wang

Multi-agent systems represent a significant advancement in artificial intelligence, enabling complex problem-solving through coordinated specialized agents. However, these systems face fundamental challenges in context management,…

Multiagent Systems · Computer Science 2025-05-01 Naveen Krishnan

The Model Context Protocol (MCP) has unified the interface between Large Language Models (LLMs) and external tools, yet a fundamental gap remains in how agents conceptualize the environments within which they operate. Current paradigms are…

Artificial Intelligence · Computer Science 2026-05-12 Giridhar Ganapavarapu , Dhaval Patel

The Model Context Protocol (MCP) (MCP Community, 2025) has emerged as a widely used framework for enabling LLM-based agents to communicate with external tools and services. The original MCP implementation (Anthropic, 2024) relies on a Large…

Distributed, Parallel, and Cluster Computing · Computer Science 2026-01-26 Meenakshi Amulya Jayanti , X. Y. Han

Graphical User Interface (GUI) action grounding is a critical step in GUI automation that maps language instructions to actionable elements on GUI screens. Most recent works of GUI action grounding leverage large GUI datasets to fine-tune…

Computation and Language · Computer Science 2025-01-28 Yue Fan , Handong Zhao , Ruiyi Zhang , Yu Shen , Xin Eric Wang , Gang Wu

In the field of MLLM-based GUI agents, compared to smartphones, the PC scenario not only features a more complex interactive environment, but also involves more intricate intra- and inter-app workflows. To address these issues, we propose a…

Computer Vision and Pattern Recognition · Computer Science 2025-02-24 Haowei Liu , Xi Zhang , Haiyang Xu , Yuyang Wanyan , Junyang Wang , Ming Yan , Ji Zhang , Chunfeng Yuan , Changsheng Xu , Weiming Hu , Fei Huang

The Model Context Protocol (MCP) standardizes how AI agents discover and invoke external tools, with over 10,000 active servers and 97 million monthly SDK downloads as of early 2026. Yet MCP does not yet standardize how agents safely…

Software Engineering · Computer Science 2026-04-16 Vasundra Srinivasan

Since the introduction of the Model Context Protocol (MCP), the number of available tools for Large Language Models (LLMs) has increased significantly. These task-specific tool sets offer an alternative to general-purpose tools such as web…

Computation and Language · Computer Science 2025-12-12 Reza Esfandiarpoor , Vishwas Suryanarayanan , Stephen H. Bach , Vishal Chowdhary , Anthony Aue

Large language model (LLM)-powered agents are increasingly used to plan and execute scientific workflows, yet most research cyberinfrastructure (CI) exposes heterogeneous APIs and implements security models that present barriers for use by…

While large language model--powered agents can self-evolve by accumulating experience or by dynamically creating new assets (i.e., tools or expert agents), existing frameworks typically treat these two evolutionary processes in isolation.…

Computation and Language · Computer Science 2026-04-14 Zihao Cheng , Zeming Liu , Yingyu Shan , Xinyi Wang , Xiangrong Zhu , Yunpu Ma , Hongru Wang , Yuhang Guo , Wei Lin , Yunhong Wang

The Model Context Protocol (MCP) replaces static, developer-controlled API integrations with more dynamic, user-driven agent systems, which also introduces new security risks. As MCP adoption grows across community servers and major…

Cryptography and Security · Computer Science 2025-11-27 Herman Errico , Jiquan Ngiam , Shanita Sojan

GUI automation faces critical challenges in dynamic environments. MLLMs suffer from two key issues: misinterpreting UI components and outdated knowledge. Traditional fine-tuning methods are costly for app-specific knowledge updates. We…

Artificial Intelligence · Computer Science 2025-05-23 Bin Xie , Rui Shao , Gongwei Chen , Kaiwen Zhou , Yinchuan Li , Jie Liu , Min Zhang , Liqiang Nie

External knowledge has played a crucial role in the recent development of computer use agents. We identify a critical knowledge-execution gap: retrieved knowledge often fails to translate into effective real-world task execution. Our…

Human-Computer Interaction · Computer Science 2025-11-04 Ziyun Zhang , Xinyi Liu , Xiaoyi Zhang , Jun Wang , Gang Chen , Yan Lu
‹ Prev 1 2 3 10 Next ›