English
Related papers

Related papers: OSExpert: Computer-Use Agents Learning Professiona…

200 papers

While scaling laws promise significant performance gains for recommender systems, efficiently deploying hyperscale models remains a major unsolved challenge. In contrast to fields where FMs are already widely adopted such as natural…

Information Retrieval · Computer Science 2025-08-08 Dai Li , Kevin Course , Wei Li , Hongwei Li , Jie Hua , Yiqi Chen , Zhao Zhu , Rui Jian , Xuan Cao , Bi Xue , Yu Shi , Jing Qian , Kai Ren , Matt Ma , Qunshu Zhang , Rui Li

Despite rapid progress in autonomous web agents, human involvement remains essential for shaping preferences and correcting agent behavior as tasks unfold. However, current agentic systems lack a principled understanding of when and why…

Computation and Language · Computer Science 2026-03-02 Faria Huq , Zora Zhiruo Wang , Zhanqiu Guo , Venu Arvind Arangarajan , Tianyue Ou , Frank Xu , Shuyan Zhou , Graham Neubig , Jeffrey P. Bigham

Graphical User Interface (GUI) agents have the potential to assist users in interacting with complex software (e.g., PowerPoint, Photoshop). While prior research has primarily focused on automating user actions through clicks and…

Computer Vision and Pattern Recognition · Computer Science 2026-03-30 Saelyne Yang , Jaesang Yu , Yi-Hao Peng , Kevin Qinghong Lin , Jae Won Cho , Yale Song , Juho Kim

The recent surge in AI agents that autonomously communicate, collaborate with humans and use diverse tools has unlocked promising opportunities in various real-world settings. However, a vital aspect remains underexplored: how agents handle…

Machine Learning · Computer Science 2025-11-04 Nabeel Seedat , Jiashuo Liu , Mihaela van der Schaar

We present HippoCamp, a new benchmark designed to evaluate agents' capabilities on multimodal file management. Unlike existing agent benchmarks that focus on tasks like web interaction, tool use, or software automation in generic settings,…

Artificial Intelligence · Computer Science 2026-04-02 Zhe Yang , Shulin Tian , Kairui Hu , Shuai Liu , Hoang-Nhat Nguyen , Yichi Zhang , Zujin Guo , Mengying Yu , Zinan Zhang , Jingkang Yang , Chen Change Loy , Ziwei Liu

Recently, large reasoning models have demonstrated strong mathematical and coding abilities, and deep search leverages their reasoning capabilities in challenging information retrieval tasks. Existing deep search works are generally limited…

Information Retrieval · Computer Science 2025-08-12 Jiejun Tan , Zhicheng Dou , Yan Yu , Jiehan Cheng , Qiang Ju , Jian Xie , Ji-Rong Wen

Large language model (LLM) agents currently depend on predefined tools or early-stage tool generation, limiting their adaptability and scalability to complex scientific tasks. We introduce CASCADE, a self-evolving agentic framework…

Artificial Intelligence · Computer Science 2026-01-29 Xu Huang , Junwu Chen , Yuxing Fei , Zhuohan Li , Philippe Schwaller , Gerbrand Ceder

LLMs have made significant progress in complex but easy-to-verify problems, yet they still struggle with discovering the unknown. In this paper, we present \textbf{AlphaResearch}, an autonomous research agent designed to discover new…

Computation and Language · Computer Science 2026-04-02 Zhaojian Yu , Kaiyue Feng , Yilun Zhao , Shilin He , Xiao-Ping Zhang , Arman Cohan

Computer use agents represent an emerging area in artificial intelligence, aiming to operate computers autonomously to fulfill user tasks, attracting significant attention from both industry and academia. However, the performance of…

Artificial Intelligence · Computer Science 2026-01-23 Yuhao Cheng , Liang Tang , Shuxian Li , Yukang Huo , Tiaonan Duan , Kaer Huang , Yanzhe Jing , Yiqiang Yan

Recent advances in vision-language models (VLMs) and reinforcement learning (RL) have driven progress in GUI automation. However, most existing methods rely on static, one-shot visual inputs and passive perception, lacking the ability to…

Artificial Intelligence · Computer Science 2026-01-16 Chen Chen , Jiawei Shao , Dakuan Lu , Haoyi Hu , Xiangcheng Liu , Hantao Yao , Wu Liu

Computer-use agents (CUAs) automate on-screen work, as illustrated by GPT-5.4 and Claude. Yet their reliability on complex, low-frequency interactions is still poor, limiting user trust. Our analysis of failure cases from advanced models…

Computer Vision and Pattern Recognition · Computer Science 2026-05-13 Miaosen Zhang , Xiaohan Zhao , Zhihong Tan , Zhou Huoshen , Yijia Fan , Yifan Yang , Kai Qiu , Bei Liu , Justin Wagle , Chenzhong Yin , Mingxi Cheng , Ji Li , Qi Dai , Chong Luo , Xu Yang , Xin Geng , Baining Guo

Conversational search presents opportunities to support users in their search activities to improve the effectiveness and efficiency of search while reducing their cognitive load. Limitations of the potential competency of conversational…

Human-Computer Interaction · Computer Science 2021-04-12 Abhishek Kaushik , Gareth J. F. Jones

Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning and tool use. However, the fundamental cognitive faculties essential for problem solving, including perception, reasoning, and memory, remain the stable…

Artificial Intelligence · Computer Science 2026-05-12 Xiaozhe Li , Jixuan Chen , Xinyu Fang , Shengyuan Ding , Haodong Duan , Qingwen Liu , Kai Chen

We develop a general problem setting for training and testing the ability of agents to gather information efficiently. Specifically, we present a collection of tasks in which success requires searching through a partially-observed…

Machine Learning · Computer Science 2016-12-09 Philip Bachman , Alessandro Sordoni , Adam Trischler

Online lifelong learning enables agents to accumulate experience across interactions and continually improve on long-horizon tasks. However, existing methods typically treat retrieval from past experience as a passive operation, triggering…

Computation and Language · Computer Science 2026-04-23 Yuxuan Cai , Jie Zhou , Qin Chen , Liang He

Although exploratory behaviors are ubiquitous in the animal kingdom, their computational underpinnings are still largely unknown. Behavioral Psychology has identified learning as a primary drive underlying many exploratory behaviors.…

Machine Learning · Computer Science 2011-12-14 Daniel Y. Little , Friedrich T. Sommer

Different activation functions work best for different deep learning models. To exploit this, we leverage recent advancements in gradient-based search techniques for neural architectures to efficiently identify high-performing activation…

Machine Learning · Computer Science 2024-08-14 Lukas Strack , Mahmoud Safari , Frank Hutter

An agent, artificial or human, must be continuously adjusting its behaviour in order to thrive in a more or less demanding environment. An artificial agent with the ability to predict human emotional valence in a geospatial and temporal…

Artificial Intelligence · Computer Science 2020-03-10 Nuno A. C. Henriques , Helder Coelho , Leonel Garcia-Marques

While current Computer Use Agent (CUA) benchmarks measure task completion effectively, they provide limited assessment of enterprise deployment readiness, emphasizing functional correctness over the operational reliability required for…

Software Engineering · Computer Science 2025-11-24 Horia Cristescu , Charles Park , Trong Canh Nguyen , Sergiu Talmacel , Alexandru-Gabriel Ilie , Stefan Adam

The last few years have witnessed substantial progress in the field of embodied AI where artificial agents, mirroring biological counterparts, are now able to learn from interaction to accomplish complex tasks. Despite this success,…

Computer Vision and Pattern Recognition · Computer Science 2022-01-04 Sarah Pratt , Luca Weihs , Ali Farhadi