English
Related papers

Related papers: Willful Disobedience: Automatically Detecting Fail…

200 papers

As LLM-based agents increasingly rely on external tools, it is important to evaluate their ability to sustain tool-grounded reasoning beyond familiar workflows and short-range interactions. We introduce AgentEscapeBench, an…

Artificial Intelligence · Computer Science 2026-05-21 Zhengkang Guo , Yiyang Li , Lin Qiu , Xiaohua Wang , Jingwen Xv , Dongyu Ru , Xiaoyu Li , Xiaoqing Zheng , Xuezhi Cao , Xunliang Cai

Graphical User Interface (GUI) agents can automate complex tasks across digital environments, but their development is hindered by the scarcity of high-quality trajectory data for training. Existing approaches rely on expensive human…

Computation and Language · Computer Science 2025-03-04 Yiheng Xu , Dunjie Lu , Zhennan Shen , Junli Wang , Zekun Wang , Yuchen Mao , Caiming Xiong , Tao Yu

As autonomous AI agents are increasingly deployed in high-stakes environments, ensuring their safety and alignment with human values is becoming a practical deployment concern. Current benchmarks for AI agents primarily evaluate refusal of…

Artificial Intelligence · Computer Science 2026-05-12 Miles Q. Li , Benjamin C. M. Fung , Martin Weiss , Pulei Xiong , Khalil Al-Hussaeni , Claude Fachkha

AI agents are autonomous systems that combine LLMs with external tools to solve complex tasks. While such tools extend capability, improper tool permissions introduce security risks such as indirect prompt injection and tool misuse. We…

Cryptography and Security · Computer Science 2026-01-21 Roy Betser , Shamik Bose , Amit Giloni , Chiara Picardi , Sindhu Padakandla , Roman Vainshtein

Autonomous AI agents are deployed at unprecedented scale, yet no principled methodology exists for verifying that an agent has not regressed after changes to its prompts, tools, models, or orchestration logic. We present AgentAssay, the…

Artificial Intelligence · Computer Science 2026-03-04 Varun Pratap Bhardwaj

Interactive agent benchmarks map an agent run to a binary outcome through outcome checks. When these checks rely on surface level signals or fail to capture the agent's actual action path, they cannot reliably determine whether the run…

Artificial Intelligence · Computer Science 2026-05-12 Shanshan Gao , Liyi Zhou

Agentic AI systems plan, use tools, maintain state, and act across multi-step workflows with external effects, meaning trustworthy deployment can no longer be judged by task completion alone. The current literature remains fragmented across…

Software Engineering · Computer Science 2026-04-23 Christopher Koch , Joshua Andreas Wellbrock

Agents built on LLMs are increasingly deployed across diverse domains, automating complex decision-making and task execution. However, their autonomy introduces safety risks, including security vulnerabilities, legal violations, and…

Artificial Intelligence · Computer Science 2025-08-01 Haoyu Wang , Christopher M. Poskitt , Jun Sun

We present AgentOptics, an agentic AI framework for high-fidelity, autonomous optical system control built on the Model Context Protocol (MCP). AgentOptics interprets natural language tasks and executes protocol-compliant actions on…

In recent years, machine learning (ML) based software systems are increasingly deployed in several critical applications, yet systematic testing of their behavior remains challenging due to complex model architectures, large input spaces,…

Software Engineering · Computer Science 2026-03-17 Fadel Mamar Seydou , Arnab Sharma

Code agents are advancing rapidly, but debugging them is becoming increasingly difficult. As frameworks orchestrate parallel tool calls and multi-stage workflows over complex tasks, making the agent's state transitions and error propagation…

Video production workflows offer a rich and demanding arena for evaluating multimodal AI agents: they require composite capabilities across text, image, audio, and video understanding, along with long-horizon planning, and tool use. To this…

Cryptography and Security · Computer Science 2026-05-28 Zongheng Cao , Yi Zheng , Rui Song , Xinyu Hu

AI agents are increasingly deployed to execute important tasks. While rising accuracy scores on standard benchmarks suggest rapid progress, many agents still continue to fail in practice. This discrepancy highlights a fundamental limitation…

Artificial Intelligence · Computer Science 2026-02-24 Stephan Rabanser , Sayash Kapoor , Peter Kirgis , Kangheng Liu , Saiteja Utpala , Arvind Narayanan

As autonomous coding agents become deeply embedded in software development workflows, their high operational velocity introduces a critical oversight challenge: the accumulating divergence between agentic actions and architectural intent.…

Software Engineering · Computer Science 2026-05-05 Matteo Casserini , Alessandro Facchini , Andrea Ferrario

LLM-based agents have demonstrated promising adaptability in real-world applications. However, these agents remain vulnerable to a wide range of attacks, such as tool poisoning and malicious instructions, that compromise their execution…

Cryptography and Security · Computer Science 2025-10-14 Jiahao Liu , Bonan Ruan , Xianglin Yang , Zhiwei Lin , Yan Liu , Yang Wang , Tao Wei , Zhenkai Liang

Traditional software relies on contracts -- APIs, type systems, assertions -- to specify and enforce correct behavior. AI agents, by contrast, operate on prompts and natural language instructions with no formal behavioral specification.…

Artificial Intelligence · Computer Science 2026-02-27 Varun Pratap Bhardwaj

Significant digitalization of financial services in a short period of time has led to an urgent demand to have autonomous, transparent and real-time credit risk decision making systems. The traditional machine learning models are effective…

Artificial Intelligence · Computer Science 2026-01-06 Chandra Sekhar Kubam

This paper develops a control-theoretic framework for analyzing agentic systems embedded within feedback control loops, where an AI agent may adapt controller parameters, select among control strategies, invoke external tools, reconfigure…

Systems and Control · Electrical Eng. & Systems 2026-03-26 Ali Eslami , Jiangbo Yu

Static benchmarks measure what AI agents can do at a fixed point in time but not how they are adopted, maintained, or experienced in deployment. We introduce AgentPulse, a continuous evaluation framework scoring 50 agents across 10 workload…

Artificial Intelligence · Computer Science 2026-04-28 Yuxuan Gao , Megan Wang , Yi Ling Yu

Agentic AI coding systems can inspect repositories, plan implementation steps, edit files, call tools, run tests, and submit pull requests. These capabilities make software and hardware development faster in some settings, but current…

Software Engineering · Computer Science 2026-05-21 Christopher Koch