English

The Predictive-Causal Gap: An Impossibility Theorem and Large-Scale Neural Evidence

Machine Learning 2026-05-07 v1

Abstract

We report a systematic failure mode in predictive representation learning. Across 2695 neural network configurations trained to predict linear-Gaussian dynamics, the optimal encoder tracks the environment rather than the system it is meant to model. The mean causal fidelity -- the fraction of encoder sensitivity allocated to system degrees of freedom -- is 0.49, and only 2.5% of configurations exceed 0.70. The failure intensifies with dimension: at N=100, the optimal encoder becomes causally blind (fidelity ~10^{-8}) while achieving 92% lower prediction error than the causal representation. We prove this is not an optimization artifact but a structural property of the predictive objective: when environment modes are slower or less noisy than system modes, every minimizer of the population risk encodes the former. The set of dynamics exhibiting this predictive-causal gap is open and of positive measure in parameter space. In a nonlinear Duffing-GRU sweep, unconstrained predictors learn environment-dominant representations in 55% of tasks (95% CI 41--68%) versus 24% under operational grounding (p=2.3e-3); the median out-of-distribution MSE inflation under environment shift is 1.82x versus 1.00x. Operational grounding -- restricting the loss to system observables -- partially suppresses the gap, but causal fidelity is never recovered without an explicit system-environment boundary. The results identify the predictive-causal gap as a structural limit of learning, with implications for self-supervised representation learning, world models, and the scaling paradigm.

Keywords

Cite

@article{arxiv.2605.05029,
  title  = {The Predictive-Causal Gap: An Impossibility Theorem and Large-Scale Neural Evidence},
  author = {Kejun Liu},
  journal= {arXiv preprint arXiv:2605.05029},
  year   = {2026}
}

Comments

15 pages, 5 figures, 3 tables. Supplemental Material included (Sections S1-S10)

R2 v1 2026-07-01T12:52:59.591Z