English
Related papers

Related papers: PAGER: Bridging the Semantic-Execution Gap in Poin…

200 papers

Recent advances in open-vocabulary mobile manipulation have brought robots into real domestic environments. In such settings, reliable long-horizon execution under open-set object references and frequent disturbances becomes essential.…

Robotics · Computer Science 2026-04-29 Jinhao Jiang , Shengyu Fang , Sibo Zuo , Yujie Tang , Yirui Li

General-purpose automated software engineering (ASE) includes tasks such as code completion, retrieval, repair, QA, and summarization. These tasks require a code retrieval system that can handle specific queries about code entities, or code…

Software Engineering · Computer Science 2025-10-01 Pratik Shah , Rajat Ghosh , Aryan Singhal , Debojyoti Dutta

Successfully solving long-horizon manipulation tasks remains a fundamental challenge. These tasks involve extended action sequences and complex object interactions, presenting a critical gap between high-level symbolic planning and…

Robotics · Computer Science 2025-09-29 Jialiang Li , Wenzheng Wu , Gaojing Zhang , Yifan Han , Wenzhao Lian

Graphical user interfaces (GUI) automation agents are emerging as powerful tools, enabling humans to accomplish increasingly complex tasks on smart devices. However, users often inadvertently omit key information when conveying tasks, which…

Computer Vision and Pattern Recognition · Computer Science 2025-08-26 Ziming Cheng , Zhiyuan Huang , Junting Pan , Zhaohui Hou , Mingjie Zhan

Web agents, which couple language models with browsing and tool-use capabilities, show promise as open web assistants. Yet progress is increasingly limited by the lack of scalable, process-level supervision. Existing benchmarks are largely…

Artificial Intelligence · Computer Science 2026-05-29 Tenghao Huang , Kung-Hsiang Huang , Prafulla Kumar Choubey , Yilun Zhou , Muhao Chen , Jonathan May , Chien-Sheng Wu

High-speed multi-agent autonomous racing demands robust spatiotemporal planning and precise control under strict computational limits. Current methods often oversimplify interactions or abandon strict kinematic constraints. We resolve this…

Robotics · Computer Science 2026-03-11 Mingyi Zhang , Cheng Hu , Yiqin Wang , Haotong Qin , Hongye Su , Lei Xie

Current mobile GUI agent benchmarks systematically fail to assess memory capabilities, with only 5.2-11.8% memory-related tasks and no cross-session learning evaluation. We introduce MemGUI-Bench, a comprehensive memory-centric benchmark…

Distributed, Parallel, and Cluster Computing · Computer Science 2026-02-09 Guangyi Liu , Pengxiang Zhao , Yaozhen Liang , Qinyi Luo , Shunye Tang , Yuxiang Chai , Weifeng Lin , Han Xiao , WenHao Wang , Siheng Chen , Zhengxi Lu , Gao Wu , Hao Wang , Liang Liu , Yong Liu

Large language models (LLMs) have shown that generative pretraining can distill vast world knowledge into compact token representations. While LLMs encapsulate extensive world knowledge, they remain limited in modeling the behavioral…

Machine Learning · Computer Science 2026-03-31 Guilin Li , Yun Zhang , Xiuyuan Chen , Chengqi Li , Bo Wang , Linghe Kong , Wenjia Wang , Weiran Huang , Matthias Hwai Yong Tan

Automation of existing Graphical User Interfaces (GUIs) is important but hard to achieve. Upstream of making the GUI user-accessible or somehow scriptable, even the data-collection to understand the original interface poses significant…

Human-Computer Interaction · Computer Science 2025-04-15 Iason Chaimalas , Arnas Vyšniauskas , Gabriel Brostow

Computer-use agents provide a promising path toward general software automation because they can interact directly with arbitrary graphical user interfaces instead of relying on brittle, application-specific integrations. Despite recent…

Artificial Intelligence · Computer Science 2026-05-01 Jinbiao Wei , Kangqi Ni , Yilun Zhao , Guo Gan , Arman Cohan

Learning visuomotor policies from scarce expert demonstrations remains a core challenge in robotic manipulation. A primary hurdle lies in distilling high-dimensional RGB representations into control-relevant geometry without overfitting.…

Robotics · Computer Science 2026-05-18 Davide Buoso , Andrea Protopapa , Stefano Di Carlo , Francesca Pistilli , Giuseppe Averta

Inter-object relations underpin spatial intelligence, yet existing representations -- linguistic prepositions or object-level scene graphs -- are too coarse to specify which regions actually support, contain, or contact one another, leading…

Computer Vision and Pattern Recognition · Computer Science 2026-03-10 Yinuo Bai , Peijun Xu , Kuixiang Shao , Yuyang Jiao , Jingxuan Zhang , Kaixin Yao , Jiayuan Gu , Jingyi Yu

Large Language Models (LLMs) have shown impressive capabilities in downstream software engineering tasks such as Automated Program Repair (APR). In particular, there has been a lot of research on repository-level issue-resolution benchmarks…

Software Engineering · Computer Science 2025-06-23 Anvith Pabba , Alex Mathai , Anindya Chakraborty , Baishakhi Ray

In the field of MLLM-based GUI agents, compared to smartphones, the PC scenario not only features a more complex interactive environment, but also involves more intricate intra- and inter-app workflows. To address these issues, we propose a…

Computer Vision and Pattern Recognition · Computer Science 2025-02-24 Haowei Liu , Xi Zhang , Haiyang Xu , Yuyang Wanyan , Junyang Wang , Ming Yan , Ji Zhang , Chunfeng Yuan , Changsheng Xu , Weiming Hu , Fei Huang

Computer-Use Agents (CUA) are becoming increasingly capable of autonomously operating digital environments through Graphical User Interfaces (GUI). Yet, most GUI remain designed primarily for humans--prioritizing aesthetics and…

Computer Vision and Pattern Recognition · Computer Science 2025-11-20 Kevin Qinghong Lin , Siyuan Hu , Linjie Li , Zhengyuan Yang , Lijuan Wang , Philip Torr , Mike Zheng Shou

Smartphone GUI agents execute tasks by operating directly on app interfaces, offering a path to broad capability without deep system integration. However, real-world smartphone use is highly personalized: users adopt diverse workflows and…

Artificial Intelligence · Computer Science 2026-04-01 Hongyi Nie , Xunyuan Liu , Yudong Bai , Yaqing Wang , Yang Liu , Quanming Yao , Zhen Wang

While GUI agents have made significant progress in web navigation and basic operating system tasks, their capabilities in professional creative workflows remain largely underexplored. To bridge this gap, we introduce Cutverse, a benchmark…

Computer Vision and Pattern Recognition · Computer Science 2026-05-20 Haobo Hu , Xiangwu Guo , Zhiheng Chen , Difei Gao , Haotian Liu , Libiao Jin , Qi Mao

Pixel-based reinforcement learning agents often fail under purely visual distribution shift even when latent dynamics and rewards are unchanged, but existing benchmarks entangle multiple sources of shift and hinder systematic analysis. We…

Machine Learning · Computer Science 2026-01-21 Egor Cherepanov , Daniil Zelezetsky , Alexey K. Kovalev , Aleksandr I. Panov

Multimodal Large Language Models (MLLMs) have significantly advanced GUI agents, yet long-horizon automation remains constrained by two critical bottlenecks: context overload from raw sequential trajectory dependence and architectural…

Artificial Intelligence · Computer Science 2026-04-15 Weihua Cheng , Junming Liu , Yifei Sun , Botian Shi , Yirong Chen , Ding Wang

Multi-agent systems (MAS) powered by large language models (LLMs) have emerged as a powerful paradigm for complex problem solving, where performance critically depends on the underlying inter-agent communication topology. However, existing…

Machine Learning · Computer Science 2026-05-19 Xuefei Wang , Jialu Wang , Fengbo Zhang , Yihan Hu , Di Zhang , Yutong Ye , Yikun Ban , Jun Han , Ruijie Wang