English
Related papers

Related papers: EE-MCP: Self-Evolving MCP-GUI Agents via Automated…

200 papers

Model Context Protocol (MCP) has recently gained increased attention within the AI community for providing a standardized way for large language models (LLMs) to interact with external tools and services, significantly enhancing their…

Distributed, Parallel, and Cluster Computing · Computer Science 2025-11-12 Zihao Ding , Mufeng Zhu , Yao Liu

Building autonomous agents that perceive and operate graphical user interfaces (GUIs) like humans has long been a vision in the field of artificial intelligence. Central to these agents is the capability for GUI interaction, which involves…

Computer Vision and Pattern Recognition · Computer Science 2025-09-23 Hongxin Li , Jingran Su , Jingfan Chen , Zheng Ju , Yuntao Chen , Qing Li , Zhaoxiang Zhang

Large language model powered autonomous agents demand robust, standardized protocols to integrate tools, share contextual data, and coordinate tasks across heterogeneous systems. Ad-hoc integrations are difficult to scale, secure, and…

Artificial Intelligence · Computer Science 2025-05-26 Abul Ehtesham , Aditi Singh , Gaurav Kumar Gupta , Saket Kumar

Large-scale, high-quality interaction trajectories are essential for advancing mobile Graphical User Interface (GUI) agents. While existing methods typically rely on labor-intensive human demonstrations or automated model exploration to…

Artificial Intelligence · Computer Science 2026-02-02 Linjia Kang , Zhimin Wang , Yongkang Zhang , Duo Wu , Jinghe Wang , Ming Ma , Haopeng Yan , Zhi Wang

High-quality dialogue is crucial for e-commerce customer service, yet traditional intent-based systems struggle with dynamic, multi-turn interactions. We present MindFlow+, a self-evolving dialogue agent that learns domain-specific behavior…

Computation and Language · Computer Science 2025-07-28 Ming Gong , Xucheng Huang , Ziheng Xu , Vijayan K. Asari

Large Language Models (LLMs) have evolved into AI agents that interact with external tools and environments to perform complex tasks. The Model Context Protocol (MCP) has become the de facto standard for connecting agents with such…

Cryptography and Security · Computer Science 2026-04-27 Christoph Bühler , Matteo Biagiola , Luca Di Grazia , Guido Salvaneschi

The development of autonomous agents increasingly relies on Multimodal Language Models (MLMs) to perform tasks described in natural language with GUI environments, such as websites, desktop computers, or mobile phones. Existing benchmarks…

Embodied agents are expected to operate persistently in dynamic physical environments, continuously acquiring new capabilities over time. Existing approaches to improving agent performance often rely on modifying the agent itself -- through…

Robotics · Computer Science 2026-05-22 Xue Qin , Simin Luan , John See , Cong Yang , Zhijun Li

As Agentic AI gain mainstream adoption, the industry invests heavily in model capabilities, achieving rapid leaps in reasoning and quality. However, these systems remain largely confined to data silos, and each new integration requires…

Cryptography and Security · Computer Science 2025-05-20 Sonu Kumar , Anubhav Girdhar , Ritesh Patil , Divyansh Tripathi

Automation of existing Graphical User Interfaces (GUIs) is important but hard to achieve. Upstream of making the GUI user-accessible or somehow scriptable, even the data-collection to understand the original interface poses significant…

Human-Computer Interaction · Computer Science 2025-04-15 Iason Chaimalas , Arnas Vyšniauskas , Gabriel Brostow

Human-AI collaboration faces growing challenges as AI systems increasingly outperform humans on complex tasks, while humans remain responsible for orchestration, validation, and decision oversight. To address this imbalance, we introduce…

Human-Computer Interaction · Computer Science 2026-02-16 Yuanrong Tang , Huiling Peng , Bingxi Zhao , Hengyang Ding , Hanchao Song , Tianhong Wang , Chen Zhong , Jiangtao Gong

LLM-powered tool-calling agents fulfill user requests by interacting with environments, querying data, and invoking tools in a multi-turn process. Yet, most existing benchmarks evaluate these systems under static environment interfaces,…

Designing high-performance system heuristics is a creative, iterative process requiring experts to form hypotheses and execute multi-step conceptual shifts. While Large Language Models (LLMs) show promise in automating this loop, they…

Artificial Intelligence · Computer Science 2026-03-24 Pantea Karimi , Kimia Noorbakhsh , Mohammad Alizadeh , Hari Balakrishnan

The increased adoption of the Model Context Protocol (MCP) for AI Agents necessitates robust security for Enterprise integrations. This paper introduces the MCP Gateway to simplify self-hosted MCP server integration. The proposed…

Cryptography and Security · Computer Science 2025-04-29 Ivo Brett

Large Language Models (LLMs) are increasingly augmented with external tools through standardized interfaces like the Model Context Protocol (MCP). However, current MCP implementations face critical limitations: they typically require local…

Cryptography and Security · Computer Science 2026-03-11 Arash Ahmadi , Sarah Sharif , Yaser M. Banad

Large Language Model (LLM)-based agents have recently shown impressive capabilities in complex reasoning and tool use via multi-step interactions with their environments. While these agents have the potential to tackle complicated tasks,…

Artificial Intelligence · Computer Science 2025-11-04 Jiaye Lin , Yifu Guo , Yuzhen Han , Sen Hu , Ziyi Ni , Licheng Wang , Mingguang Chen , Hongzhang Liu , Ronghao Chen , Yangfan He , Daxin Jiang , Binxing Jiao , Chen Hu , Huacan Wang

Modern GUI agents typically rely on a model-centric and step-wise interaction paradigm, where LLMs must re-interpret the UI and re-decide actions at every screen, which is fragile in long-horizon tasks. In this paper, we propose Executable…

Artificial Intelligence · Computer Science 2026-05-13 Zerui Qin , Sheng Yue , Xingyuan Hua , Yongjian Fu , Ju Ren

Current approaches to AI agent orchestration typically involve building multi-agent frameworks that manage context passing, memory, error handling, and step coordination through code. These frameworks work well for complex, concurrent…

Artificial Intelligence · Computer Science 2026-03-19 Jake Van Clief , David McDermott

Most commodity software lacks accessible Application Programming Interfaces (APIs), requiring autonomous agents to interact solely through pixel-based Graphical User Interfaces (GUIs). In this API-free setting, large language model…

Artificial Intelligence · Computer Science 2026-03-26 Chenwei Tang , Lin Long , Xinyu Liu , Jingyu Xing , Zizhou Wang , Joey Tianyi Zhou , Jiawei Du , Liangli Zhen , Jiancheng Lv

Mobile GUI agents excel at immediate reactive control but frequently fail in realistic, long-horizon tasks that require memory. This failure stems from a fundamental conflict between limited context windows and token-heavy screenshots. To…

Computation and Language · Computer Science 2026-05-29 Junyang Wang , Haiyang Xu , Xi Zhang , Zhaoqing Zhu , Ming Yan , Jieping Ye , Jitao Sang