English

When and How Long? The Readout-Mediator Angle in Temporal Reasoning

Machine Learning 2026-05-29 v1 Artificial Intelligence

Abstract

A linear probe can decode a representation almost perfectly and yet be completely irrelevant to how the model uses it. On calendar-date duration reasoning in language models, a sin\sin/cos\cos probe recovers day-of-year from a layer's activations, yet ablating its direction has no effect on the model's answers -- while ablating a four-dimensional subspace found by Distributed Alignment Search (DAS) at the same layer collapses performance entirely. We measure the angle between these two subspaces -- the \emph{readout-mediator angle} -- and find it indistinguishable from the angle between two random subspaces (the Haar-uniform null), meaning the probe has learned a direction orthogonal to the model's actual computation. Reverse-engineering the circuit reveals why: attention heads route month-grained context through learned QK offsets at ±30{\pm}30 and ±61{\pm}61 days, and MLPs then convert \emph{when} (absolute date) into \emph{how long} (duration) -- all downstream of the causal subspace the probe never touches. Sparse-autoencoder decomposition confirms the split: probe-aligned and DAS-aligned features encode semantically disjoint concepts with negligible causal overlap. The dissociation replicates across four scales (1.51.5-99\,B) and two model families, with preliminary evidence on two further domains (spatial displacement, symbolic arithmetic), suggesting that readout-mediator orthogonality is a general failure mode of probe-based interpretability. This directly undermines proposals to deploy probes as runtime safety monitors: the probe can report high confidence on a direction the model has silently abandoned.

Cite

@article{arxiv.2605.29126,
  title  = {When and How Long? The Readout-Mediator Angle in Temporal Reasoning},
  author = {Shreyas Fadnavis and Praitayini Kanakaraj and Felix Wyss},
  journal= {arXiv preprint arXiv:2605.29126},
  year   = {2026}
}
R2 v1 2026-07-22T07:38:18.712Z