English
Related papers

Related papers: MAGNET: Towards Adaptive GUI Agents with Memory-Dr…

200 papers

Although numerous strategies have recently been proposed to enhance the autonomous interaction capabilities of multimodal agents in graphical user interface (GUI), their reliability remains limited when faced with complex or out-of-domain…

Computation and Language · Computer Science 2025-10-06 Pengzhou Cheng , Lingzhong Dong , Zeng Wu , Zongru Wu , Xiangru Tang , Chengwei Qin , Zhuosheng Zhang , Gongshen Liu

Memory-Augmented Generation (MAG) extends Large Language Models with external memory to support long-context reasoning, but existing approaches largely rely on semantic similarity over monolithic memory stores, entangling temporal, causal,…

Artificial Intelligence · Computer Science 2026-04-17 Dongming Jiang , Yi Li , Guanpeng Li , Bingzhe Li

Multimodal Large Language Model (MLLM) agents facilitate Graphical User Interface (GUI) automation but struggle with long-horizon, cross-application tasks due to limited context windows. While memory systems provide a viable solution,…

Artificial Intelligence · Computer Science 2026-02-02 Hongze Mi , Yibo Feng , WenJie Lu , Song Cao , Jinyuan Li , Yanming Li , Xuelin Zhang , Haotian Luo , Songyang Peng , He Cui , Tengfei Tian , Jun Fang , Hua Chai , Naiqiang Tan

Mobile agents are essential for automating tasks in complex and dynamic mobile environments. As foundation models evolve, the demands for agents that can adapt in real-time and process multimodal data have grown. This survey provides a…

Artificial Intelligence · Computer Science 2025-09-16 Biao Wu , Yanda Li , Zhiwei Zhang , Yunchao Wei , Meng Fang , Ling Chen

The development of artificial intelligence systems is transitioning from creating static, task-specific models to dynamic, agent-based systems capable of performing well in a wide range of applications. We propose an Interactive Agent…

Large Language Models (LLMs) based agents excel at diverse tasks, yet they suffer from brittle procedural memory that is manually engineered or entangled in static parameters. In this work, we investigate strategies to endow agents with a…

Computation and Language · Computer Science 2026-04-16 Runnan Fang , Yuan Liang , Xiaobin Wang , Jialong Wu , Shuofei Qiao , Pengjun Xie , Fei Huang , Huajun Chen , Ningyu Zhang

Despite the potential of language model-based agents to solve real-world tasks such as web navigation, current methods still struggle with long-horizon tasks with complex action trajectories. In contrast, humans can flexibly solve complex…

Computation and Language · Computer Science 2024-09-12 Zora Zhiruo Wang , Jiayuan Mao , Daniel Fried , Graham Neubig

The recent advancement of autonomous agents powered by Large Language Models (LLMs) has demonstrated significant potential for automating tasks on mobile devices through graphical user interfaces (GUIs). Despite initial progress, these…

Human-Computer Interaction · Computer Science 2025-07-30 Yi Kong , Dianxi Shi , Guoli Yang , Zhang ke-di , Chenlin Huang , Xiaopeng Li , Songchang Jin

Graphical user interface (GUI) agents are rapidly progressing toward autonomous interaction and reliable task execution across diverse applications. However, two central challenges remain unresolved: automating the evaluation of agent…

Large Language Model (LLM)-based mobile agents have made significant performance advancements. However, these agents often follow explicit user instructions while overlooking personalized needs, leading to significant limitations for real…

Computation and Language · Computer Science 2026-01-29 Shuoxin Wang , Chang Liu , Gowen Loo , Lifan Zheng , Kaiwen Wei , Xinyi Zeng , Jingyuan Zhang , Yu Tian

LLM-powered embodied agents have shown success on conventional object-rearrangement tasks, but providing personalized assistance that leverages user-specific knowledge from past interactions presents new challenges. We investigate these…

Computation and Language · Computer Science 2026-02-16 Taeyoon Kwon , Dongwook Choi , Hyojun Kim , Sunghwan Kim , Seungjun Moon , Beong-woo Kwak , Kuan-Hao Huang , Jinyoung Yeo

The remarkable progress of vision-language models (VLMs) has enabled GUI agents to interact with computers in a human-like manner. Yet real-world computer-use tasks remain difficult due to long-horizon workflows, diverse interfaces, and…

Artificial Intelligence · Computer Science 2026-03-12 Sibo Zhu , Wenyi Wu , Kun Zhou , Stephen Wang , Biwei Huang

Large Language Models have demonstrated remarkable capabilities across diverse domains, yet significant challenges persist when deploying them as AI agents for real-world long-horizon tasks. Existing LLM agents suffer from a critical…

Computation and Language · Computer Science 2025-10-10 Cheng Yang , Xuemeng Yang , Licheng Wen , Daocheng Fu , Jianbiao Mei , Rong Wu , Pinlong Cai , Yufan Shen , Nianchen Deng , Botian Shi , Yu Qiao , Haifeng Li

Modern GUI agents typically rely on a model-centric and step-wise interaction paradigm, where LLMs must re-interpret the UI and re-decide actions at every screen, which is fragile in long-horizon tasks. In this paper, we propose Executable…

Artificial Intelligence · Computer Science 2026-05-13 Zerui Qin , Sheng Yue , Xingyuan Hua , Yongjian Fu , Ju Ren

With the growing adoption of large language model agents in persistent real-world roles, they naturally encounter continuous streams of tasks. A key limitation, however, is their failure to learn from the accumulated interaction history,…

Graphical User Interface (GUI) Agents, powered by multimodal large language models (MLLMs), have shown great potential for task automation on computing devices such as computers and mobile phones. However, existing agents face challenges in…

Artificial Intelligence · Computer Science 2025-01-09 Yuhang Liu , Pengxiang Li , Zishu Wei , Congkai Xie , Xueyu Hu , Xinchen Xu , Shengyu Zhang , Xiaotian Han , Hongxia Yang , Fei Wu

Large Language Model (LLM) agents are increasingly deployed to automate complex workflows in mobile and desktop environments. However, current model-centric agent architectures struggle to self-evolve post-deployment: improving…

Artificial Intelligence · Computer Science 2025-12-19 Zibin Liu , Cheng Zhang , Xi Zhao , Yunfei Feng , Bingyu Bai , Dahu Feng , Erhu Feng , Yubin Xia , Haibo Chen

Self-evolving multi-agent systems (MAS) have emerged as a promising route to LLM agents that continually improve from experience, with persistent memory at their foundation. However, existing designs almost exclusively adopt a centralized…

Multiagent Systems · Computer Science 2026-05-22 Guangya Hao , Yunbo Long , Zhuokai Zhao

Graphical User Interface (GUI) agents, driven by Multi-modal Large Language Models (MLLMs), have emerged as a promising paradigm for enabling intelligent interaction with digital systems. This paper provides a structured survey of recent…

Artificial Intelligence · Computer Science 2025-05-14 Jiahao Li , Kaer Huang

In this work, we propose MetaAgent, an agentic paradigm inspired by the principle of learning-by-doing, where expertise is developed through hands-on practice and continual self-improvement. MetaAgent starts with a minimal workflow,…

Artificial Intelligence · Computer Science 2025-09-03 Hongjin Qian , Zheng Liu