English
Related papers

Related papers: ReInAgent: A Context-Aware GUI Agent Enabling Huma…

200 papers

Large reasoning models have demonstrated strong problem-solving abilities, yet real-world tasks often require external tools and long-horizon interactions. Existing agent frameworks typically follow predefined workflows, which limit…

Artificial Intelligence · Computer Science 2026-02-06 Xiaoxi Li , Wenxiang Jiao , Jiarui Jin , Guanting Dong , Jiajie Jin , Yinuo Wang , Hao Wang , Yutao Zhu , Ji-Rong Wen , Yuan Lu , Zhicheng Dou

Autonomous Graphical User Interface (GUI) agents often struggle with multi-step tasks due to constrained context windows and static policies that fail to adapt to dynamic environments. To address these limitations, this work proposes the…

Machine Learning · Computer Science 2026-05-19 Shilong Jin , Lanjun Wang , Zhuosheng Zhang

We are exploring the enhancement of models of agent behaviour with more "human-like" decision making strategies than are presently available. Our motivation is to developed with a view to as the decision analysis and support for electric…

Multiagent Systems · Computer Science 2009-12-22 Yee Ming Chen , Bo-Yuan Wang , Hung-Ming Shiu

Large language models (LLMs) and agent-based frameworks have advanced rapidly, enabling diverse applications. Yet, with the proliferation of models and agentic strategies, practitioners face substantial uncertainty in selecting the best…

Computation and Language · Computer Science 2025-10-08 Zheyuan Zhang , Kaiwen Shi , Zhengqing Yuan , Zehong Wang , Tianyi Ma , Keerthiram Murugesan , Vincent Galassi , Chuxu Zhang , Yanfang Ye

Artificial Intelligence (AI) agents capable of autonomous learning and independent decision-making hold great promise for addressing complex challenges across various critical infrastructure domains, including transportation, energy…

Multiagent Systems · Computer Science 2025-07-02 Hepeng Li , Yuhong Liu , Jun Yan , Jie Gao , Xiaoou Yang

AI agents powered by large language models are increasingly capable of autonomously completing complex, multi-step tasks using external tools. Yet, they still fall short of human-level performance in most domains including computer use,…

Building generalist agents that can rapidly adapt to new environments is a key challenge for deploying AI in the digital and real worlds. Is scaling current agent architectures the most effective way to build generalist agents? We propose a…

Artificial Intelligence · Computer Science 2025-02-25 Kaustubh Sridhar , Souradeep Dutta , Dinesh Jayaraman , Insup Lee

The Graphical User Interface (GUI) is how users interact with mobile apps. To ensure it functions properly, testing engineers have to make sure it functions as intended, based on test requirements that are typically written in natural…

Software Engineering · Computer Science 2024-07-15 Yongxiang Hu , Xuan Wang , Yingchuan Wang , Yu Zhang , Shiyu Guo , Chaoyi Chen , Xin Wang , Yangfan Zhou

We introduce a novel large language model (LLM)-driven agent framework, which iteratively refines queries and filters contextual evidence by leveraging dynamically evolving knowledge. A defining feature of the system is its decoupling of…

Artificial Intelligence · Computer Science 2025-04-02 Seyoung Song

While Graphical User Interface (GUI) agents have shown promising performance in automated device interaction, they primarily depend on static parametric knowledge from pre-training or instruction tuning. This reliance fundamentally limits…

Artificial Intelligence · Computer Science 2026-05-19 Jingjing Liu , Ziye Huang , Zihao Cheng , Zeming Liu , Jiahong Wu , Yuhang Guo , Kehai Chen , Yunhong Wang , Haifeng Wang

Controlling desktop applications via software remains a fundamental yet under-served problem. Existing multi-modal large language models (MLLMs) ingest screenshots and task instructions to generate keystrokes and mouse events, but they…

Artificial Intelligence · Computer Science 2025-09-24 Zihan Dong , Xinyu Fan , Zixiang Tang , Yunqing Li

Recent advances in autonomous digital agents from industry (e.g., Manus AI and Gemini's research mode) highlight potential for structured tasks by autonomous decision-making and task decomposition; however, it remains unclear to what extent…

Artificial Intelligence · Computer Science 2025-09-03 Yen-Che Chien , Kuang-Da Wang , Wei-Yao Wang , Wen-Chih Peng

Large Language Models (LLMs) have the capacity of performing complex scheduling in a multi-agent system and can coordinate these agents into completing sophisticated tasks that require extensive collaboration. However, despite the…

Artificial Intelligence · Computer Science 2023-09-20 Ran Gong , Qiuyuan Huang , Xiaojian Ma , Hoi Vo , Zane Durante , Yusuke Noda , Zilong Zheng , Song-Chun Zhu , Demetri Terzopoulos , Li Fei-Fei , Jianfeng Gao

Large Language Model (LLM) empowered agents have recently emerged as advanced paradigms that exhibit impressive capabilities in a wide range of domains and tasks. Despite their potential, current LLM agents often adopt a one-size-fits-all…

Agentic systems, AI architectures that autonomously execute multi-step workflows to achieve complex goals, are often built using repeated large language model (LLM) calls for closed-set decision tasks such as routing, shortlisting, gating,…

Computation and Language · Computer Science 2026-02-19 Ido Levy , Eilam Shapira , Yinon Goldshtein , Avi Yaeli , Nir Mashkif , Segev Shlomov

Most web agents operate at the human interface level, observing screenshots or raw DOM trees without application-level access, which limits robustness and action expressiveness. In enterprise settings, however, explicit control of both the…

Artificial Intelligence · Computer Science 2026-02-17 Chenyang Ma , Clyde Fare , Matthew Wilson , Dave Braines

Effective knowledge management is critical for preserving institutional expertise and improving the efficiency of workforce training in state transportation agencies. Traditional approaches, such as static documentation, classroom-based…

Computation and Language · Computer Science 2026-03-05 Divija Amaram , Lu Gao , Gowtham Reddy Gudla , Tejaswini Sanjay Katale

Mobile GUI agents are becoming critical tools to improve user experience on smart devices, with multimodal large language models (MLLMs) emerging as the dominant paradigms in this domain. Current agents, however, rely on explicit human…

Human-Computer Interaction · Computer Science 2026-03-17 Qinglong Yang , Haoming Li , Haotian Zhao , Xiaokai Yan , Jingtao Ding , Fengli Xu , Yong Li

We present SmartWalkCoach, a mobile AI companion that supports the full walking journey: from pre-walk planning to in-walk guidance through to post-walk reflection. Addressing a gap between map navigation and motivational coaching,…

Human-Computer Interaction · Computer Science 2026-05-15 Xianzhe Zhang , Mingxuan Hu , Bufan Xue , Erick Purwanto , Thomas J Selig , Daniel Yonto

We present Agent S, an open agentic framework that enables autonomous interaction with computers through a Graphical User Interface (GUI), aimed at transforming human-computer interaction by automating complex, multi-step tasks. Agent S…

Artificial Intelligence · Computer Science 2024-10-11 Saaket Agashe , Jiuzhou Han , Shuyu Gan , Jiachen Yang , Ang Li , Xin Eric Wang