English
Related papers

Related papers: ReInAgent: A Context-Aware GUI Agent Enabling Huma…

200 papers

Personalized mobile agents that infer user preferences and calibrate proactive assistance hold great promise as everyday digital assistants, yet existing benchmarks fail to capture what this requires. Prior work evaluates preference…

Despite improvements by length extrapolation, efficient attention and memory modules, handling infinitely long documents with linear complexity without performance degradation during extrapolation remains the ultimate challenge in long-text…

Computation and Language · Computer Science 2025-07-04 Hongli Yu , Tinghong Chen , Jiangtao Feng , Jiangjie Chen , Weinan Dai , Qiying Yu , Ya-Qin Zhang , Wei-Ying Ma , Jingjing Liu , Mingxuan Wang , Hao Zhou

Large Language Models demonstrate outstanding performance in many language tasks but still face fundamental challenges in managing the non-linear flow of human conversation. The prevalent approach of treating dialogue history as a flat,…

Computation and Language · Computer Science 2026-04-20 Junan Hu , Shudan Guo , Wenqi Liu , Jianhua Yin , Yinwei Wei

Existing Multimodal Large Language Model (MLLM)-based agents face significant challenges in handling complex GUI (Graphical User Interface) interactions on devices. These challenges arise from the dynamic and structured nature of GUI…

Web-based platforms are becoming a primary channel for psychological support, yet most LLM-driven chatbots remain opaque, single-stage, and weakly grounded in established therapeutic practice, limiting their usefulness for web applications…

Human-Computer Interaction · Computer Science 2026-03-10 Fei Wang , Jiangnan Yang , Junjie Chen , Yuxin Liu , Kun Li , Yanyan Wei , Dan Guo , Meng Wang

While GUI agents have shown strong performance under explicit and completion instructions, real-world deployment requires aligning with users' more complex implicit intents. In this work, we highlight Hierarchical Implicit Intent Alignment…

Artificial Intelligence · Computer Science 2026-05-14 Yibo Lyu , Gongwei Chen , Rui Shao , Weili Guan , Liqiang Nie

Building agents that autonomously operate mobile devices has attracted increasing attention. While Vision-Language Models (VLMs) show promise, most existing approaches rely on direct state-to-action mappings, which lack structured reasoning…

Artificial Intelligence · Computer Science 2026-02-09 Zhe Wu , Hongjin Lu , Junliang Xing , Changhao Zhang , Yuxuan Li , Yin Zhu , Yuhao Yang , Yuheng Jing , Kai Li , Kun Shao , Jianye Hao , Jun Wang , Yuanchun Shi

Reliability is key to realizing the promise of autonomous UI-Agents, multimodal agents that directly interact with apps in the same manner as humans, as users must be able to trust an agent to complete a given task. Current evaluations rely…

Web agents based on large language models have demonstrated promising capability in automating web tasks. However, current web agents struggle to reason out sensible actions due to the limitations of predicting environment changes, and…

Artificial Intelligence · Computer Science 2026-02-18 Zhouzhou Shen , Xueyu Hu , Xiyun Li , Tianqing Fang , Juncheng Li , Shengyu Zhang

Standard reinforcement learning (RL) for large language model (LLM) agents typically optimizes extrinsic rewards, prioritizing isolated task completion over continual adaptation. Consequently, agents often converge to suboptimal policies…

Artificial Intelligence · Computer Science 2026-03-31 Xiaoying Zhang , Zichen Liu , Yipeng Zhang , Xia Hu , Wenqi Shao

Responsible AI has risen to the forefront of the AI research community. As neural network-based learning algorithms continue to permeate real-world applications, the field of Responsible AI has played a large role in ensuring that such…

Artificial Intelligence · Computer Science 2023-11-06 Niko A. Grupen

Recent years have witnessed a rapid development of mobile GUI agents powered by large language models (LLMs), which can autonomously execute diverse device-control tasks based on natural language instructions. The increasing accuracy of…

Cryptography and Security · Computer Science 2026-04-15 Guohong Liu , Jialei Ye , Jiacheng Liu , Yuanchun Li , Wei Liu , Pengzhi Gao , Jian Luan , Yunxin Liu

As global tourism expands and artificial intelligence technology advances, intelligent travel planning services have emerged as a significant research focus. Within dynamic real-world travel scenarios with multi-dimensional constraints,…

Artificial Intelligence · Computer Science 2024-09-13 Aili Chen , Xuyang Ge , Ziquan Fu , Yanghua Xiao , Jiangjie Chen

Recent advancements in Large Generative Models (LGMs) have revolutionized multi-modal generation. However, generating illustrated storybooks remains an open challenge, where prior works mainly decompose this task into separate stages, and…

Computer Vision and Pattern Recognition · Computer Science 2026-04-21 Bo Gao , Chang Liu , Yuyang Miao , Siyuan Ma , Ser-Nam Lim

Intelligent agent systems in real-world agricultural scenarios must handle diverse tasks under multimodal inputs, ranging from lightweight information understanding to complex multi-step execution. However, most existing approaches rely on…

Computation and Language · Computer Science 2026-01-14 Bo Yang , Yu Zhang , Yunkui Chen , Lanfei Feng , Xiao Xu , Nueraili Aierken , Shijian Li

For machine agents to successfully interact with humans in real-world settings, they will need to develop an understanding of human mental life. Intuitive psychology, the ability to reason about hidden mental variables that drive observable…

Multi-agent systems provide a powerful way to extend large language models (LLMs) by decomposing a complex task into specialized subtasks handled by different agents. However, their performance is often hindered by error propagation,…

Machine Learning · Computer Science 2026-05-14 Zheng Wang , Yuang Liu , Yangkai Ding

Current benchmarks for graphical user interface (GUI) agents predominantly rely on static screenshots. However, real-world smartphone interaction routinely requires agents to process transient audio cues and temporal video dynamics that are…

Human-Computer Interaction · Computer Science 2026-05-20 Felix Henry , Xiaochen Lin , Jiangyou Zhu , Yangfan , Bingqian Zhang , Min Chen , Shiyu Huang

Proactive AR agents promise context-aware assistance, but their interactions often rely on explicit voice prompts or responses, which can be disruptive or socially awkward. We introduce Sensible Agent, a framework designed for unobtrusive…

Human-Computer Interaction · Computer Science 2025-11-03 Geonsun Lee , Min Xia , Nels Numan , Xun Qian , David Li , Yanhe Chen , Achin Kulshrestha , Ishan Chatterjee , Yinda Zhang , Dinesh Manocha , David Kim , Ruofei Du

We introduce DriveAgent, a novel multi-agent autonomous driving framework that leverages large language model (LLM) reasoning combined with multimodal sensor fusion to enhance situational understanding and decision-making. DriveAgent…

Robotics · Computer Science 2025-05-06 Xinmeng Hou , Wuqi Wang , Long Yang , Hao Lin , Jinglun Feng , Haigen Min , Xiangmo Zhao