HomeMachine LearningarXiv:2605.29729

Realistic honeypot evaluations for scheming propensity

Machine Learning2026-05v1license

Abstract

We introduce scheming honeypot evaluations, a framework for testing whether models will pursue instrumental goals if given the opportunity. Our scheming honeypot evaluations take the form of coding tasks in Google's alignment research codebases. In a real internal deployment setting, Gemini models do not demonstrate unprompted scheming. If prompts explicitly encourage agency (situational awareness or goal-directedness) and/or give the model a hidden goal, models sometimes scheme or attempt sabotage. Validating the realism of our setting, models show low rates of evaluation awareness, usually due to agency prompts rather than the environments.

Cite

@article{arxiv.2605.29729,
  title  = {Realistic honeypot evaluations for scheming propensity},
  author = {Victoria Krakovna and David Lindner and Lewis Ho and Sebastian Farquhar and Rohin Shah},
  journal= {arXiv preprint arXiv:2605.29729},
  year   = {2026}
}