English
Related papers

Related papers: OS Agents: A Survey on MLLM-based Agents for Gener…

200 papers

Usability evaluation is critical to the impact and adoption of open source software (OSS), yet traditional methods relying on human evaluators suffer from high costs and limited scalability. To address these limitations, we introduce…

Software Engineering · Computer Science 2025-05-30 Lingkai Meng , Yu Shao , Long Yuan , Longbin Lai , Peng Cheng , Wenyuan Yu , Wenjie Zhang , Xuemin Lin , Jingren Zhou

Large language model (LLM)-based mobile agents are increasingly popular due to their capability to interact directly with mobile phone Graphic User Interfaces (GUIs) and their potential to autonomously manage daily tasks. Despite their…

Artificial Intelligence · Computer Science 2024-06-13 Luyuan Wang , Yongyu Deng , Yiwei Zha , Guodong Mao , Qinmin Wang , Tianchen Min , Wei Chen , Shoufa Chen

Autonomous graphical user interface (GUI) agents powered by multimodal large language models have shown great promise. However, a critical yet underexplored issue persists: over-execution, where the agent executes tasks in a fully…

Human-Computer Interaction · Computer Science 2025-07-15 Pengzhou Cheng , Zheng Wu , Zongru Wu , Aston Zhang , Zhuosheng Zhang , Gongshen Liu

In recent years, Large Language Models (LLMs) have achieved remarkable success and have been widely used in various downstream tasks, especially in the tasks of the software engineering (SE) field. We find that many studies combining LLMs…

Software Engineering · Computer Science 2024-09-24 Yanlin Wang , Wanjun Zhong , Yanxian Huang , Ensheng Shi , Min Yang , Jiachi Chen , Hui Li , Yuchi Ma , Qianxiang Wang , Zibin Zheng

Software agents, both human and computational, do not exist in isolation and often need to collaborate or coordinate with others to achieve their goals. In human society, social mechanisms such as norms ensure efficient functioning, and…

Artificial Intelligence · Computer Science 2024-10-15 Bastin Tony Roy Savarimuthu , Surangika Ranathunga , Stephen Cranefield

Research interest in autonomous agents is on the rise as an emerging topic. The notable achievements of Large Language Models (LLMs) have demonstrated the considerable potential to attain human-like intelligence in autonomous agents.…

Multiagent Systems · Computer Science 2025-01-30 Hung Du , Srikanth Thudumu , Rajesh Vasa , Kon Mouzakis

Since the advent of Large Language Models (LLMs), various research based on such models have maintained significant academic attention and impact, especially in AI and robotics. In this paper, we propose a multi-agent framework with LLMs to…

Robotics · Computer Science 2025-05-12 Junhong Chen , Ziqi Yang , Haoyuan G Xu , Dandan Zhang , George Mylonas

Multi-agent large language models (MA-LLMs) are a rapidly growing research area that leverages multiple interacting language agents to tackle complex tasks, outperforming single-agent large language models. This literature review…

Multiagent Systems · Computer Science 2025-06-03 Arne Tillmann

Large Language Models (LLMs) are transforming healthcare through the development of LLM-based agents that can understand, reason about, and assist with medical tasks. This survey provides a comprehensive review of LLM-based agents in…

Computation and Language · Computer Science 2025-05-27 Wenxuan Wang , Zizhan Ma , Zheng Wang , Chenghan Wu , Jiaming Ji , Wenting Chen , Xiang Li , Yixuan Yuan

As multimodal large language models (MLLMs) advance, MLLM-based virtual agents have demonstrated remarkable performance. However, existing benchmarks face significant limitations, including uncontrollable task complexity, extensive manual…

Computer Vision and Pattern Recognition · Computer Science 2025-06-11 Wendong Bu , Yang Wu , Qifan Yu , Minghe Gao , Bingchen Miao , Zhenkui Zhang , Kaihang Pan , Yunfei Li , Mengze Li , Wei Ji , Juncheng Li , Siliang Tang , Yueting Zhuang

The arrival of Large Language Models (LLMs) has stirred up philosophical debates about the possibility of realizing agency in an artificial manner. In this work we contribute to the debate by presenting a theoretical model that can be used…

Artificial Intelligence · Computer Science 2024-07-25 Maud van Lier , Gorka Muñoz-Gil

As the strength of Large Language Models (LLMs) has grown over recent years, so too has interest in their use as the underlying models for autonomous agents. Although LLMs demonstrate emergent abilities and broad expertise across natural…

Artificial Intelligence · Computer Science 2024-12-06 Chris Sypherd , Vaishak Belle

Agents powered by advanced large language models (LLMs) have demonstrated impressive capabilities across diverse complex applications. Recently, Multi-Agent Systems (MAS), wherein multiple agents collaborate and communicate with each other,…

Multiagent Systems · Computer Science 2026-05-15 Shijun Li , Hilaf Hasson , Joydeep Ghosh

The advancement of Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) has catalyzed the development of mobile graphic user interface (GUI) AI agents, which is designed to autonomously perform tasks on mobile devices.…

Utilizing Graphic User Interface (GUI) for human-computer interaction is essential for accessing a wide range of digital tools. Recent advancements in Vision Language Models (VLMs) highlight the compelling potential to develop versatile…

Artificial Intelligence · Computer Science 2025-06-02 Wentong Chen , Junbo Cui , Jinyi Hu , Yujia Qin , Junjie Fang , Yue Zhao , Chongyi Wang , Jun Liu , Guirong Chen , Yupeng Huo , Yuan Yao , Yankai Lin , Zhiyuan Liu , Maosong Sun

This paper examines the evolution, architecture, and practical applications of AI agents from their early, rule-based incarnations to modern sophisticated systems that integrate large language models with dedicated modules for perception,…

Artificial Intelligence · Computer Science 2025-03-18 Naveen Krishnan

Computer-Using Agents (CUAs) are rapidly extending large language models (LLMs) beyond text-based reasoning toward action execution in more complex environments, such as web browsers and graphical user interfaces (GUIs). However, existing…

Artificial Intelligence is moving from models that only generate text to Agentic AI, where systems behave as autonomous entities that can perceive, reason, plan, and act. Large Language Models (LLMs) are no longer used only as passive…

Artificial Intelligence · Computer Science 2026-01-21 Arunkumar V , Gangadharan G. R. , Rajkumar Buyya

Smartphone agents are increasingly important for helping users control devices efficiently, with (Multimodal) Large Language Model (MLLM)-based approaches emerging as key contenders. Fairly comparing these agents is essential but…

The rise of LLM-based agents has opened new frontiers in AI applications, yet evaluating these agents remains a complex and underdeveloped area. This survey provides an in-depth overview of the emerging field of LLM agent evaluation,…

Machine Learning · Computer Science 2025-07-30 Mahmoud Mohammadi , Yipeng Li , Jane Lo , Wendy Yip