English
Related papers

Related papers: Measuring Harmfulness of Computer-Using Agents

200 papers

AI leaders and safety reports increasingly warn that advances in model reasoning may enable biological misuse, including by low-expertise users, while major labs describe safeguards as expanding but still evolving rather than settled. This…

Computers and Society · Computer Science 2026-04-24 Michael Richter

Large Language Models (LLMs) have the potential to significantly enhance threat intelligence by automating the collection, preprocessing, and analysis of threat data. However, the usability of these tools is critical to ensure their…

Cryptography and Security · Computer Science 2024-09-24 Sanchana Srikanth , Mohammad Hasanuzzaman , Farah Tasnur Meem

Personal AI agents like OpenClaw run with elevated privileges on users' local machines, where a single successful prompt injection can leak credentials, redirect financial transactions, or destroy files. This threat goes well beyond…

Artificial Intelligence · Computer Science 2026-04-07 Bowen Wei , Yunbei Zhang , Jinhao Pan , Kai Mei , Xiao Wang , Jihun Hamm , Ziwei Zhu , Yingqiang Ge

Computer-use agents (CUAs) promise to automate complex tasks across operating systems (OS) and the web, but remain vulnerable to indirect prompt injection. Current evaluations of this threat either lack support realistic but controlled…

Computation and Language · Computer Science 2026-03-03 Zeyi Liao , Jaylen Jones , Linxi Jiang , Yuting Ning , Eric Fosler-Lussier , Yu Su , Zhiqiang Lin , Huan Sun

Large language models (LLMs) introduce new security risks, but there are few comprehensive evaluation suites to measure and reduce these risks. We present BenchmarkName, a novel benchmark to quantify LLM security risks and capabilities. We…

Computer-use agents (CUAs) automate on-screen work, as illustrated by GPT-5.4 and Claude. Yet their reliability on complex, low-frequency interactions is still poor, limiting user trust. Our analysis of failure cases from advanced models…

Computer Vision and Pattern Recognition · Computer Science 2026-05-13 Miaosen Zhang , Xiaohan Zhao , Zhihong Tan , Zhou Huoshen , Yijia Fan , Yifan Yang , Kai Qiu , Bei Liu , Justin Wagle , Chenzhong Yin , Mingxi Cheng , Ji Li , Qi Dai , Chong Luo , Xu Yang , Xin Geng , Baining Guo

Recent advances in AI agents capable of solving complex, everyday tasks, from scheduling to customer service, have enabled deployment in real-world settings, but their possibilities for unsafe behavior demands rigorous evaluation. While…

Artificial Intelligence · Computer Science 2026-02-18 Sanidhya Vijayvargiya , Aditya Bharat Soni , Xuhui Zhou , Zora Zhiruo Wang , Nouha Dziri , Graham Neubig , Maarten Sap

Large Language Models (LLMs) have been increasingly integrated into computer-use agents, which can autonomously operate tools on a user's computer to accomplish complex tasks. However, due to the inherently unstable and unpredictable nature…

Cryptography and Security · Computer Science 2025-09-10 Haitao Hu , Peng Chen , Yanpeng Zhao , Yuqi Chen

AI systems have become increasingly capable of dangerous behaviours in many domains. This raises the question: Do models sometimes choose to violate human instructions in order to perform behaviour that is more useful for certain goals? We…

Artificial Intelligence · Computer Science 2026-05-08 Jonas Wiedermann-Möller , Leonard Dung , Maksym Andriushchenko

Healthcare administration accounts for over $1 trillion in annual spending, making it a promising target for LLM-based computer-use agents (CUAs). While clinical applications of LLMs have received significant attention, no benchmark exists…

Large language model (LLM) agents are increasingly capable of autonomously conducting cyberattacks, posing significant threats to existing applications. This growing risk highlights the urgent need for a real-world benchmark to evaluate the…

Computer-use agents (CUAs) have made tremendous progress in the past year, yet they still frequently produce misaligned actions that deviate from the user's original intent. Such misaligned actions may arise from external attacks (e.g.,…

Computation and Language · Computer Science 2026-02-10 Yuting Ning , Jaylen Jones , Zhehao Zhang , Chentao Ye , Weitong Ruan , Junyi Li , Rahul Gupta , Huan Sun

Language Model (LM) agents for cybersecurity that are capable of autonomously identifying vulnerabilities and executing exploits have potential to cause real-world impact. Policymakers, model providers, and researchers in the AI and…

Recent advances in Language Model (LM) agents and tool use, exemplified by applications like ChatGPT Plugins, enable a rich set of capabilities but also amplify potential risks - such as leaking private data or causing financial losses.…

Artificial Intelligence · Computer Science 2024-05-20 Yangjun Ruan , Honghua Dong , Andrew Wang , Silviu Pitis , Yongchao Zhou , Jimmy Ba , Yann Dubois , Chris J. Maddison , Tatsunori Hashimoto

Graphical user interface (GUI) agents powered by vision language models (VLMs) are rapidly moving from passive assistance to autonomous operation. However, this unrestricted action space exposes users to severe and irreversible financial,…

Machine Learning · Computer Science 2026-04-13 Yushi Feng , Junye Du , Qifan Wang , Zizhan Ma , Qian Niu , Yutaka Matsuo , Long Feng , Lequan Yu

Although computer-use agents (CUAs) hold significant potential to automate increasingly complex OS workflows, they can demonstrate unsafe unintended behaviors that deviate from expected outcomes even under benign input contexts. However,…

Computation and Language · Computer Science 2026-02-10 Jaylen Jones , Zhehao Zhang , Yuting Ning , Eric Fosler-Lussier , Pierre-Luc St-Charles , Yoshua Bengio , Dawn Song , Yu Su , Huan Sun

Hacking poses a significant threat to cybersecurity, inflicting billions of dollars in damages annually. To mitigate these risks, ethical hacking, or penetration testing, is employed to identify vulnerabilities in systems and networks.…

Cryptography and Security · Computer Science 2025-02-24 Isamu Isozaki , Manil Shrestha , Rick Console , Edward Kim

Large foundation models are integrated into Computer Use Agents (CUAs), enabling autonomous interaction with operating systems through graphical user interfaces (GUIs) to perform complex tasks. This autonomy introduces serious security…

Artificial Intelligence · Computer Science 2026-01-21 Wenqi Zhang , Yulin Shen , Changyue Jiang , Jiarun Dai , Geng Hong , Xudong Pan

The rapid proliferation of LLM-based autonomous agents in real operating system environments introduces a new category of safety risk beyond content safety: behavior jailbreak, where an adversary induces an agent to execute dangerous…

Cryptography and Security · Computer Science 2026-05-12 Chiyu Zhang , Huiqin Yang , Bendong Jiang , Xiaolei Zhang , Yiran Zhao , Ruyi Chen , Lu Zhou , Xiaogang Xu , Jiafei Wu , Liming Fang , Zhe Liu

LLM-based coding agents are rapidly being deployed in software development, yet their safety implications remain poorly understood. These agents, while capable of accelerating software development, may exhibit unsafe behaviors during normal…

Artificial Intelligence · Computer Science 2025-08-26 Matous Kozak , Roshanak Zilouchian Moghaddam , Siva Sivaraman