English
Related papers

Related papers: $\pi$-Bench: Evaluating Proactive Personal Assista…

200 papers

One of the long-standing aspirations in conversational AI is to allow them to autonomously take initiatives in conversations, i.e., being proactive. This is especially challenging for multi-party conversations. Prior NLP research focused…

Human-Computer Interaction · Computer Science 2025-02-19 Xingyu Bruce Liu , Shitao Fang , Weiyan Shi , Chien-Sheng Wu , Takeo Igarashi , Xiang Anthony Chen

We introduce a new benchmark, LLF-Bench (Learning from Language Feedback Benchmark; pronounced as "elf-bench"), to evaluate the ability of AI agents to interactively learn from natural language feedback and instructions. Learning from…

Artificial Intelligence · Computer Science 2023-12-14 Ching-An Cheng , Andrey Kolobov , Dipendra Misra , Allen Nie , Adith Swaminathan

Real-time conversational assistants for procedural tasks often depend on video input, which can be computationally expensive and compromise user privacy. For the first time, we propose a real-time conversational assistant that provides…

Multimedia · Computer Science 2026-02-18 Rehana Mahfuz , Yinyi Guo , Erik Visser , Phanidhar Chinchili

Amongst the most common use cases of modern AI is LLM chat with web search enabled. However, no direct evaluations of the quality of web research agents exist that control for the continually-changing web. We introduce Deep Research Bench,…

Artificial Intelligence · Computer Science 2025-06-10 FutureSearch , : , Nikos I. Bosse , Jon Evans , Robert G. Gambee , Daniel Hnyk , Peter Mühlbacher , Lawrence Phillips , Dan Schwarz , Jack Wildman

The evolution of Large Language Models (LLMs) into autonomous agents necessitates the management of extensive, dynamic contexts. Current benchmarks, however, remain largely static, relying on passive retrieval tasks that fail to simulate…

Computation and Language · Computer Science 2026-02-02 Shicheng Fang , Yuxin Wang , Xiaoran Liu , Jiahao Lu , Chuanyuan Tan , Xinchi Chen , Yining Zheng , Xuanjing Huang , Xipeng Qiu

Entrepreneurship requires navigating open-ended, ill-defined problems: identifying risks, challenging assumptions, and making strategic decisions under deep uncertainty. Novice founders often struggle with these metacognitive demands, while…

Human-Computer Interaction · Computer Science 2025-08-18 Evey Jiaxin Huang , Matthew Easterday , Elizabeth Gerber

As agentic AI systems increasingly operate autonomously, establishing trust through verifiable evaluation becomes critical. Yet existing benchmarks lack the transparency and auditability needed to assess whether agents behave reliably. We…

Computation and Language · Computer Science 2025-12-02 Hyunjun Kim , Sooyoung Ryu

AI agents may be able to automate your inbox, but can they automate other routine aspects of your life? Everyday online tasks offer a realistic yet unsolved testbed for evaluating the next generation of AI agents. To this end, we introduce…

User models in information retrieval rest on a foundational assumption that observed behavior reveals intent. This assumption collapses when the user is an AI agent privately configured by a human operator. For any action an agent takes, a…

Agency, the capacity to proactively shape events, is central to how humans interact and collaborate. While LLMs are being developed to simulate human behavior and serve as human-like agents, little attention has been given to the Agency…

Computation and Language · Computer Science 2024-02-09 Ashish Sharma , Sudha Rao , Chris Brockett , Akanksha Malhotra , Nebojsa Jojic , Bill Dolan

Solving expert-level multimodal tasks is a key milestone towards general intelligence. As the capabilities of multimodal large language models (MLLMs) continue to improve, evaluation of such advanced multimodal intelligence becomes…

Computer Vision and Pattern Recognition · Computer Science 2025-03-11 Yan Yang , Dongxu Li , Haoning Wu , Bei Chen , Liu Liu , Liyuan Pan , Junnan Li

As multimodal large language models (MLLMs) advance, MLLM-based virtual agents have demonstrated remarkable performance. However, existing benchmarks face significant limitations, including uncontrollable task complexity, extensive manual…

Computer Vision and Pattern Recognition · Computer Science 2025-06-11 Wendong Bu , Yang Wu , Qifan Yu , Minghe Gao , Bingchen Miao , Zhenkui Zhang , Kaihang Pan , Yunfei Li , Mengze Li , Wei Ji , Juncheng Li , Siliang Tang , Yueting Zhuang

Real-world requests to AI agents are fundamentally underspecified. Natural human communication relies on shared context and unstated constraints that speakers expect listeners to infer. Current agentic benchmarks test explicit…

Artificial Intelligence · Computer Science 2026-02-25 Ved Sirdeshmukh , Marc Wetter

Large language models (LLMs) show remarkable potential to act as computer agents, enhancing human productivity and software accessibility in multi-modal tasks that require planning and reasoning. However, measuring agent performance in…

Generative AI agents, software systems powered by Large Language Models (LLMs), are emerging as a promising approach to automate cybersecurity tasks. Among the others, penetration testing is a challenging field due to the task complexity…

Cryptography and Security · Computer Science 2024-10-29 Luca Gioacchini , Marco Mellia , Idilio Drago , Alexander Delsanto , Giuseppe Siracusano , Roberto Bifulco

AI agents deployed as persistent assistants must maintain correct beliefs as their information environment evolves. In practice, evidence is scattered across heterogeneous sources that often contradict one another, new information can…

Machine Learning · Computer Science 2026-05-19 Haonian Ji , Kaiwen Xiong , Siwei Han , Peng Xia , Shi Qiu , Yiyang Zhou , Jiaqi Liu , Jinlong Li , Bingzhou Li , Zeyu Zheng , Cihang Xie , Huaxiu Yao

Information retrieval in Large Language Models (LLMs) is increasingly recognized as intertwined with generation capabilities rather than mere lookup. While longer contexts are often assumed to improve retrieval, the effects of intra-context…

Computation and Language · Computer Science 2025-08-01 Chupei Wang , Jiaqiu Vince Sun

Large language models (LLMs) have advanced the development of various AI conversational agents, including role-playing conversational agents that mimic diverse characters and human behaviors. While prior research has predominantly focused…

Computation and Language · Computer Science 2024-08-06 Hongzhan Chen , Hehong Chen , Ming Yan , Wenshen Xu , Xing Gao , Weizhou Shen , Xiaojun Quan , Chenliang Li , Ji Zhang , Fei Huang , Jingren Zhou

Artificial intelligence systems for scientific discovery have demonstrated remarkable potential, yet existing approaches remain largely proprietary and operate in batch-processing modes requiring hours per research cycle, precluding…

Artificial Intelligence · Computer Science 2026-01-28 Lukas Weidener , Marko Brkić , Mihailo Jovanović , Ritvik Singh , Chiara Baccin , Emre Ulgac , Alex Dobrin , Aakaash Meduri

Computer-Using Agents (CUAs) are rapidly extending large language models (LLMs) beyond text-based reasoning toward action execution in more complex environments, such as web browsers and graphical user interfaces (GUIs). However, existing…