English
Related papers

Related papers: $\pi$-Bench: Evaluating Proactive Personal Assista…

200 papers

Graphical User Interface (GUI) agents, powered by Large Foundation Models, have emerged as a transformative approach to automating human-computer interaction. These agents autonomously interact with digital systems or software applications…

Intention recognition, or the ability to anticipate the actions of another agent, plays a vital role in the design and development of automated assistants that can support humans in their daily tasks. In particular, industrial settings pose…

Artificial Intelligence · Computer Science 2024-11-27 Juan Carlos Saborio , Joachim Hertzberg

Deep research agents powered by Large Language Models (LLMs) can perform multi-step reasoning, web exploration, and long-form report generation. However, most existing systems operate in an autonomous manner, assuming fully specified user…

Computation and Language · Computer Science 2026-01-13 Yingchaojie Feng , Qiang Huang , Xiaoya Xie , Zhaorui Yang , Jun Yu , Wei Chen , Anthony K. H. Tung

The evolution of large language models (LLMs) has enhanced the planning capabilities of language agents in diverse real-world scenarios. Despite these advancements, the potential of LLM-powered agents to comprehend ambiguous user…

Computation and Language · Computer Science 2024-10-03 Xuan Zhang , Yang Deng , Zifeng Ren , See-Kiong Ng , Tat-Seng Chua

Generative AI is being leveraged to solve a variety of computer-use tasks involving desktop applications. State-of-the-art systems have focused solely on improving accuracy on leading benchmarks. However, these systems are practically…

Artificial Intelligence · Computer Science 2026-05-19 Reyna Abhyankar , Qi Qi , Yiying Zhang

Interactive Artificial Intelligent(AI) assistant systems are designed to offer timely guidance to help human users to complete a variety tasks. One of the remaining challenges is to understand user's mental states during the task for more…

Computation and Language · Computer Science 2024-04-01 Megan Su , Yuwei Bao

Recent advances in AI agents capable of solving complex, everyday tasks, from scheduling to customer service, have enabled deployment in real-world settings, but their possibilities for unsafe behavior demands rigorous evaluation. While…

Artificial Intelligence · Computer Science 2026-02-18 Sanidhya Vijayvargiya , Aditya Bharat Soni , Xuhui Zhou , Zora Zhiruo Wang , Nouha Dziri , Graham Neubig , Maarten Sap

Artificial intelligence (AI) assistants are increasingly embedded in workplace tools, raising the question of how initiative-taking shapes adoption. Prior work highlights trust and expectation mismatches as barriers, but the underlying…

Human-Computer Interaction · Computer Science 2026-04-28 Dana Harari , Ofra Amir

In this report, we present ML-Dev-Bench, a benchmark aimed at testing agentic capabilities on applied Machine Learning development tasks. While existing benchmarks focus on isolated coding tasks or Kaggle-style competitions, ML-Dev-Bench…

Software Engineering · Computer Science 2025-02-20 Harshith Padigela , Chintan Shah , Dinkar Juyal

The vision of a broadly capable and goal-directed agent, such as an Internet-browsing agent in the digital world and a household humanoid in the physical world, has rapidly advanced, thanks to the generalization capability of foundation…

Machine Learning · Computer Science 2024-12-18 Yifei Zhou , Qianlan Yang , Kaixiang Lin , Min Bai , Xiong Zhou , Yu-Xiong Wang , Sergey Levine , Erran Li

Collaborative problem-solving under time pressure is common but difficult, as teams must generate ideas quickly, coordinate actions, and track progress. Generative AI offers new opportunities to assist, but we know little about how…

Human-Computer Interaction · Computer Science 2026-02-23 Anirban Mukhopadhyay , Kevin Salubre , Hifza Javed , Shashank Mehrotra , Kumar Akash

Large Language Models (LLMs) excel in traditional natural language processing tasks but struggle with problems that require complex domain-specific calculations or simulations. While equipping LLMs with external tools to build LLM-based…

Software Engineering · Computer Science 2025-06-11 Bohan Lyu , Xin Cong , Heyang Yu , Pan Yang , Yujia Qin , Yining Ye , Yaxi Lu , Zhong Zhang , Yukun Yan , Yankai Lin , Zhiyuan Liu , Maosong Sun

The ability of large language models (LLMs) to mimic human-like intelligence has led to a surge in LLM-based autonomous agents. Though recent LLMs seem capable of planning and reasoning given user instructions, their effectiveness in…

In the field of MLLM-based GUI agents, compared to smartphones, the PC scenario not only features a more complex interactive environment, but also involves more intricate intra- and inter-app workflows. To address these issues, we propose a…

Computer Vision and Pattern Recognition · Computer Science 2025-02-24 Haowei Liu , Xi Zhang , Haiyang Xu , Yuyang Wanyan , Junyang Wang , Ming Yan , Ji Zhang , Chunfeng Yuan , Changsheng Xu , Weiming Hu , Fei Huang

Today's AI models learn primarily through mimicry and refining, so it is not surprising that they struggle to solve problems beyond the limits set by existing data. To solve novel problems, agents should acquire skills for exploring and…

Artificial Intelligence · Computer Science 2026-03-25 Raj Ghugare , Roger Creus Castanyer , Catherine Ji , Kathryn Wantlin , Jin Schofield , Karthik Narasimhan , Benjamin Eysenbach

Today's AI assistants such as OpenClaw are designed to handle context effectively, making context learning an increasingly important capability for models. As these systems move beyond professional settings into everyday life, the nature of…

Robotic manipulation policies often degrade over extended horizons, yet existing benchmarks provide limited insight into why such failures occur. Most prior benchmarks are either simulation-based or report aggregate success, making it…

Robotics · Computer Science 2026-04-21 Xueyao Chen , Jingkai Jia , Tong Yang , Yibo Fu , Wei Li , Wenqiang Zhang

Large language models (LLMs) are increasingly integral as productivity assistants, but existing benchmarks fall short in rigorously evaluating their real-world instruction-following capabilities. Current benchmarks often (i) lack sufficient…

Computation and Language · Computer Science 2025-09-30 Jiho Park , Jongyoon Song , Minjin Choi , Kyuho Heo , Taehun Huh , Ji Won Kim

We introduce a multi-turn benchmark for evaluating personalised alignment in LLM-based AI assistants, focusing on their ability to handle user-provided safety-critical contexts. Our assessment of ten leading models across five scenarios…

Human-Computer Interaction · Computer Science 2025-01-31 Lize Alberts , Benjamin Ellis , Andrei Lupu , Jakob Foerster

Recent advances in Natural Language Interfaces (NLIs) and Large Language Models (LLMs) have transformed our approach to NLP tasks, shifting the focus towards a more Pragmatics-based approach. This shift enables more natural interactions…

Human-Computer Interaction · Computer Science 2024-09-23 Roderick Tabalba , Christopher J. Lee , Giorgio Tran , Nurit Kirshenbaum , Jason Leigh