English
Related papers

Related papers: Implicit Intelligence -- Evaluating Agents on What…

200 papers

One of the main research areas in Artificial Intelligence is the coding of agents (programs) which are able to learn by themselves in any situation. This means that agents must be useful for purposes other than those they were created for,…

Artificial Intelligence · Computer Science 2011-02-04 Javier Insa-Cabrera , Jose Hernandez-Orallo

When deploying autonomous agents in the real world, we need effective ways of communicating objectives to them. Traditional skill learning has revolved around reinforcement and imitation learning, each with rigid constraints on the format…

Artificial Intelligence · Computer Science 2019-11-21 Mark Woodward , Chelsea Finn , Karol Hausman

Eliciting reasoning has emerged as a powerful technique for improving the performance of large language models (LLMs) on complex tasks by inducing thinking. However, their effectiveness in realistic user-engaged agent scenarios remains…

Computation and Language · Computer Science 2026-02-10 Jiatong Li , Changdae Oh , Hyeong Kyu Choi , Jindong Wang , Sharon Li

In this preprint, we present A collaborative human-AI approach to building an inspectable semantic layer for Agentic AI. AI agents first propose candidate knowledge structures from diverse data sources; domain experts then validate,…

Artificial Intelligence · Computer Science 2025-12-05 Liam McGee , James Harvey , Lucy Cull , Andreas Vermeulen , Bart-Floris Visscher , Malvika Sharan

The increasing prevalence of Artificial Intelligence (AI) in safety-critical contexts such as air-traffic control leads to systems that are practical and efficient, and to some extent explainable to humans to be trusted and accepted. The…

Computers and Society · Computer Science 2023-06-28 Sabine Theis , Sophie Jentzsch , Fotini Deligiannaki , Charles Berro , Arne Peter Raulf , Carmen Bruder

Significant digitalization of financial services in a short period of time has led to an urgent demand to have autonomous, transparent and real-time credit risk decision making systems. The traditional machine learning models are effective…

Artificial Intelligence · Computer Science 2026-01-06 Chandra Sekhar Kubam

Natural language prompts often suffer from intent transmission loss: the gap between what users actually need and what they communicate to AI systems. We evaluate PPS (Prompt Protocol Specification), a 5W3H-based framework for structured…

Artificial Intelligence · Computer Science 2026-03-20 Peng Gang

The human-agent team, which is a problem in which humans and autonomous agents collaborate to achieve one task, is typical in human-AI collaboration. For effective collaboration, humans want to have an effective plan, but in realistic…

Artificial Intelligence · Computer Science 2021-09-02 Ryo Nakahashi , Seiji Yamada

From the earliest years of our lives, humans use language to express our beliefs and desires. Being able to talk to artificial agents about our preferences would thus fulfill a central goal of value alignment. Yet today, we lack…

Artificial Intelligence · Computer Science 2022-10-12 Theodore R Sumers , Robert D Hawkins , Mark K Ho , Thomas L Griffiths , Dylan Hadfield-Menell

AI agents that autonomously interact with external tools and environments have shown great promise across real-world applications. However, their reliance on external data exposes them to serious indirect prompt injection attacks, where…

Cryptography and Security · Computer Science 2026-05-08 Hao Li , Ruoyao Wen , Shanghao Shi , Ning Zhang , Yevgeniy Vorobeychik , Chaowei Xiao

AI systems have long been expected to interact with users, answering questions, generating content, and continuing (social) conversations. Agentic AI, however, breaks from this expectation, as its primary objective is workflow execution on…

Human-Computer Interaction · Computer Science 2026-05-05 Eunchae Jang , S. Shyam Sundar

We argue that an explainable artificial intelligence must possess a rationale for its decisions, be able to infer the purpose of observed behaviour, and be able to explain its decisions in the context of what its audience understands and…

Artificial Intelligence · Computer Science 2021-04-26 Michael Timothy Bennett , Yoshihiro Maruyama

This paper addresses the topic of robustness under sensing noise, ambiguous instructions, and human-robot interaction. We take a radically different tack to the issue of reliable embodied AI: instead of focusing on formal verification…

Robotics · Computer Science 2026-01-13 Kenneth Kwok , Basura Fernando , Qianli Xu , Vigneshwaran Subbaraju , Dongkyu Choi , Boon Kiat Quek

As large language models (LLMs) are increasingly deployed as interactive agents, open-ended human-AI interactions can involve deceptive behaviors with serious real-world consequences, yet existing evaluations remain largely…

Artificial Intelligence · Computer Science 2026-02-09 Yichen Wu , Qianqian Gao , Xudong Pan , Geng Hong , Min Yang

Intelligent agents such as robots are increasingly deployed in real-world, safety-critical settings. It is vital that these agents are able to explain the reasoning behind their decisions to human counterparts, however, their behavior is…

Machine Learning · Computer Science 2023-09-20 Xijia Zhang , Yue Guo , Simon Stepputtis , Katia Sycara , Joseph Campbell

The era of Large Language Models (LLMs) presents a new opportunity for interpretability--agentic interpretability: a multi-turn conversation with an LLM wherein the LLM proactively assists human understanding by developing and leveraging a…

Artificial Intelligence · Computer Science 2025-06-17 Been Kim , John Hewitt , Neel Nanda , Noah Fiedel , Oyvind Tafjord

As the complexity of AI systems and their interactions with the world increases, generating explanations for their behaviour is important for safely deploying AI. For agents, the most natural abstractions for predicting behaviour attribute…

Artificial Intelligence · Computer Science 2025-06-05 Alexis Bellot , Jonathan Richens , Tom Everitt

AI agents, predominantly powered by large language models (LLMs), are vulnerable to indirect prompt injection, in which malicious instructions embedded in untrusted data can trigger dangerous agent actions. This position paper discusses our…

Cryptography and Security · Computer Science 2026-04-01 Chong Xiang , Drew Zagieboylo , Shaona Ghosh , Sanjay Kariyappa , Kai Greshake , Hanshen Xiao , Chaowei Xiao , G. Edward Suh

As agentic AI systems increasingly operate autonomously, establishing trust through verifiable evaluation becomes critical. Yet existing benchmarks lack the transparency and auditability needed to assess whether agents behave reliably. We…

Computation and Language · Computer Science 2025-12-02 Hyunjun Kim , Sooyoung Ryu

Language agents have demonstrated remarkable potential in web search and information retrieval. However, these search agents assume user queries are complete and unambiguous, an assumption that diverges from reality where users begin with…