View From Above: A Framework for Evaluating Distribution Shifts in Model Behavior
Computation and Language
2024-10-01 v3 Artificial Intelligence
Machine Learning
Abstract
When large language models (LLMs) are asked to perform certain tasks, how can we be sure that their learned representations align with reality? We propose a domain-agnostic framework for systematically evaluating distribution shifts in LLMs decision-making processes, where they are given control of mechanisms governed by pre-defined rules. While individual LLM actions may appear consistent with expected behavior, across a large number of trials, statistically significant distribution shifts can emerge. To test this, we construct a well-defined environment with known outcome logic: blackjack. In more than 1,000 trials, we uncover statistically significant evidence suggesting behavioral misalignment in the learned representations of LLM.
Cite
@article{arxiv.2407.00948,
title = {View From Above: A Framework for Evaluating Distribution Shifts in Model Behavior},
author = {Tanush Chopra and Michael Li and Jacob Haimes},
journal= {arXiv preprint arXiv:2407.00948},
year = {2024}
}