Laws of thermodynamics for exponential families
Statistical Mechanics
2025-01-07 v1 Machine Learning
Statistics Theory
Statistics Theory
Abstract
We develop the laws of thermodynamics in terms of general exponential families. By casting learning (log-loss minimization) problems in max-entropy and statistical mechanics terms, we translate thermodynamics results to learning scenarios. We extend the well-known way in which exponential families characterize thermodynamic and learning equilibria. Basic ideas of work and heat, and advanced concepts of thermodynamic cycles and equipartition of energy, find exact and useful counterparts in AI / statistics terms. These ideas have broad implications for quantifying and addressing distribution shift.
Cite
@article{arxiv.2501.02071,
title = {Laws of thermodynamics for exponential families},
author = {Akshay Balsubramani},
journal= {arXiv preprint arXiv:2501.02071},
year = {2025}
}