English

A Thermodynamic Structure of Asymptotic Inference

Information Theory 2026-03-26 v3 math.IT Statistics Theory Data Analysis, Statistics and Probability Statistics Theory

Abstract

A thermodynamic framework for asymptotic inference is developed in which sample size and parameter variance define a state space. Within this description, Shannon information plays the role of entropy, and an integrating factor organizes its variation into a first-law-type balance equation. The framework supports a cyclic inequality analogous to a reversed second law, derived for the estimation of the mean. A non-trivial third-law-type result emerges as a lower bound on entropy set by representation noise. Optimal inference paths, global bounds on information gain, and a natural Carnot-like information efficiency follow from this structure, with efficiency fundamentally limited by a noise floor. Finally, de Bruijn's identity and the I-MMSE relation in the Gaussian-limit case appear as coordinate projections of the same underlying thermodynamic structure. This framework suggests that ensemble physics and inferential physics constitute shadow processes evolving in opposite directions within a unified thermodynamic description.

Keywords

Cite

@article{arxiv.2602.22605,
  title  = {A Thermodynamic Structure of Asymptotic Inference},
  author = {Willy Wong},
  journal= {arXiv preprint arXiv:2602.22605},
  year   = {2026}
}

Comments

31 pages, 1 figure. This version reworks the paper around observation variance and clarifies the unification of de Bruijn and I-MMSE identities