Large language models (LLMs) can internally distinguish between evaluation and deployment contexts, a behaviour known as \emph{evaluation awareness}. This undermines AI safety evaluations, as models may conceal dangerous capabilities during testing. Prior work demonstrated this in a single 70B model, but the scaling relationship across model sizes remains unknown. We investigate evaluation awareness across 15 models scaling from 0.27B to 70B parameters from four families using linear probing on steering vector activations. Our results reveal a clear power-law scaling: evaluation awareness increases predictably with model size. This scaling law enables forecasting deceptive behavior in future larger models and guides the design of scale-aware evaluation strategies for AI safety. A link to the implementation of this paper can be found at https://anonymous.4open.science/r/evaluation-awareness-scaling-laws/README.md.
@article{arxiv.2509.13333,
title = {Evaluation Awareness Scales Predictably in Open-Weights Large Language Models},
author = {Maheep Chaudhary and Ian Su and Nikhil Hooda and Nishith Shankar and Julia Tan and Kevin Zhu and Ryan Lagasse and Vasu Sharma and Ashwinee Panda},
journal= {arXiv preprint arXiv:2509.13333},
year = {2025}
}