English
Related papers

Related papers: Monitoring Reasoning Models for Misbehavior and th…

200 papers

Observability into the decision making of modern AI systems may be required to safely deploy increasingly capable agents. Monitoring the chain-of-thought (CoT) of today's reasoning models has proven effective for detecting misbehavior.…

Chain-of-thought (CoT) reasoning provides a significant performance uplift to LLMs by enabling planning, exploration, and deliberation of their actions. CoT is also a powerful tool for monitoring the behaviours of these agents: when…

Artificial Intelligence · Computer Science 2026-05-21 Nathaniel Mitrani Hadida , Sassan Bhanji , Cameron Tice , Puria Radmard

Chain-of-thought (CoT) monitoring is one of the most promising tools we have for detecting model misbehavior, but its effectiveness depends on models faithfully externalizing their reasoning. Motivated by this vulnerability, we study…

Machine Learning · Computer Science 2026-05-18 Reilly Haskins , Bilal Chughtai , Joshua Engels

Chain-of-Thought (CoT) monitoring has emerged as a compelling method for detecting harmful behaviors such as reward hacking for reasoning models, under the assumption that models' reasoning processes are informative of such behaviors. In…

Machine Learning · Computer Science 2026-03-10 Nikolaus Howe , Micah Carroll

Chain-of-thought (CoT) monitoring is proposed as a method for overseeing the internal reasoning of language-model agents. Prior work has shown that when models are explicitly informed that their reasoning is being monitored, or are…

Cryptography and Security · Computer Science 2026-03-19 Thomas Jiralerspong , Flemming Kondrup , Yoshua Bengio

Recent findings suggest that misaligned models may exhibit deceptive behavior, raising concerns about output trustworthiness. Chain-of-thought (CoT) is a promising tool for alignment monitoring: when models articulate their reasoning…

Cryptography and Security · Computer Science 2025-10-24 Artur Zolkowski , Wen Xing , David Lindner , Florian Tramèr , Erik Jenner

Detecting harmful AI actions is important as AI agents gain adoption. Chain-of-thought (CoT) monitoring is one method widely used to detect adversarial attacks and AI misalignment. However, attackers and misaligned models might evade CoT…

Computation and Language · Computer Science 2025-10-17 Shiyuan Guo , Henry Sleight , Fabien Roger

AI systems that output their reasoning in natural language offer an opportunity for safety -- we can \emph{monitor} their chain of thought (CoT) for undesirable reasoning, such as the pursuit of harmful objectives. However, the extent to…

Artificial Intelligence · Computer Science 2025-12-10 Matt MacDermott , Qiyao Wei , Rada Djoneva , Francis Rhys Ward

OpenAI (2025) showed that training against a chain of thought (CoT) monitor can cause obfuscated CoTs, which contain bad behavior the monitor cannot detect. They proposed to keep CoTs monitorable by training only against output monitors…

Machine Learning · Computer Science 2025-11-18 Jacob Drori , Luke Marks , Bryce Woodworth , Alex Cloud , Alexander Matt Turner

Reward hacking, where a reasoning model exploits loopholes in a reward function to achieve high rewards without solving the intended task, poses a significant threat. This behavior may be explicit, i.e. verbalized in the model's…

Artificial Intelligence · Computer Science 2026-03-03 Xinpeng Wang , Nitish Joshi , Barbara Plank , Rico Angell , He He

Chain-of-thought (CoT) monitors are LLM-based systems that analyze reasoning traces to detect when outputs may exhibit attributes of interest, such as test-hacking behavior during code generation. In this paper, we use information-theoretic…

Machine Learning · Computer Science 2026-02-23 Usman Anwar , Tim Bakker , Dana Kianfar , Cristina Pinneri , Christos Louizos

Chain-of-Thought (CoT) monitoring, in which automated systems monitor the CoT of an LLM, is a promising approach for effectively overseeing AI systems. However, the extent to which a model's CoT helps us oversee the model - the…

Machine Learning · Computer Science 2026-04-01 Max Kaufmann , David Lindner , Roland S. Zimmermann , and Rohin Shah

As AI models are deployed with increasing autonomy, it is important to ensure they do not take harmful actions unnoticed. As a potential mitigation, we investigate Chain-of-Thought (CoT) monitoring, wherein a weaker trusted monitor model…

Artificial Intelligence · Computer Science 2025-11-26 Benjamin Arnav , Pablo Bernabeu-Pérez , Nathan Helm-Burger , Tim Kostolansky , Hannes Whittingham , Mary Phuong

AI systems that "think" in human language offer a unique opportunity for AI safety: we can monitor their chains of thought (CoT) for the intent to misbehave. Like all other known AI oversight methods, CoT monitoring is imperfect and allows…

While chain-of-thought (CoT) monitoring is an appealing AI safety defense, recent work on "unfaithfulness" has cast doubt on its reliability. These findings highlight an important failure mode, particularly when CoT acts as a post-hoc…

Artificial Intelligence · Computer Science 2025-07-08 Scott Emmons , Erik Jenner , David K. Elson , Rif A. Saurous , Senthooran Rajamanoharan , Heng Chen , Irhum Shafkat , Rohin Shah

Chain-of-thought (CoT) offers a potential boon for AI safety as it allows monitoring a model's CoT to try to understand its intentions and reasoning processes. However, the effectiveness of such monitoring hinges on CoTs faithfully…

Monitoring the chain-of-thought (CoT) of reasoning models is a promising approach for detecting covert misbehavior (i.e., hidden objectives) in code generation tasks. While large models (GPT-5, Gemini-3-Flash) can serve as effective CoT…

Cryptography and Security · Computer Science 2026-05-14 Nirav Diwan , Han Wang , Berkcan Kapusuzoglu , Ramin Moradi , Supriyo Chakraborty , Giri Iyengar , Sambit Sahu , Huan Zhang , Gang Wang

Chain-of-Thought (CoT) reasoning has emerged as a key technique for eliciting complex reasoning in Large Language Models (LLMs). Although interpretable, its dependence on natural language limits the model's expressive bandwidth. Continuous…

Artificial Intelligence · Computer Science 2026-04-28 Sharan Ramjee

Modern large language models rely on chain-of-thought (CoT) reasoning to achieve impressive performance, yet the same mechanism can amplify deceptive alignment, situations in which a model appears aligned while covertly pursuing misaligned…

Artificial Intelligence · Computer Science 2025-05-27 Jiaming Ji , Wenqi Chen , Kaile Wang , Donghai Hong , Sitong Fang , Boyuan Chen , Jiayi Zhou , Juntao Dai , Sirui Han , Yike Guo , Yaodong Yang

Chain-of-thought (CoT) reasoning not only enhances large language model performance but also provides critical insights into decision-making processes, marking it as a useful tool for monitoring model intent and planning. However, recent…

‹ Prev 1 2 3 10 Next ›