Related papers: Stochastic Thermodynamics of Learning Parametric P…

We focus attention on some particular thermodynamic relations (PTR). Using information theory concepts we show that, for a reversible process, microscopic considerations related to these PTR make the concomitant informational contents of…

The issue of discrete probability estimation for samples of small size is addressed in this study. The maximum likelihood method often suffers over-fitting when insufficient data is available. Although the Bayesian approach can avoid…

We develop the laws of thermodynamics in terms of general exponential families. By casting learning (log-loss minimization) problems in max-entropy and statistical mechanics terms, we translate thermodynamics results to learning scenarios.…

Biological systems have to build models from their sensory data that allow them to efficiently process previously unseen inputs. Here, we study a neural network learning a linearly separable rule using examples provided by a teacher. We…

This paper revisits Brownian motion from the perspective of Information Theory, aiming to explore the connections between Information Theory, Thermodynamics, and Complex Science. First, we propose a single-particle discrete Brownian motion…

Large Language Models (LLMs) store an extensive amount of factual knowledge obtained from vast collections of text. To effectively utilize these models for downstream tasks, it is crucial to have reliable methods for measuring their…

For a language model (LM) to faithfully model human language, it must compress vast, potentially infinite information into relatively few dimensions. We propose analyzing compression in (pre-trained) LMs from two points of view: geometric…

With electric power systems becoming more compact and increasingly powerful, the relevance of thermal stress especially during overload operation is expected to increase ceaselessly. Whenever critical temperatures cannot be measured…

We introduce a numerical method to sample the distributions of charge, heat, and entropy production in open quantum systems coupled strongly to macroscopic reservoirs, with both temporal and energy resolution and beyond the linear-response…

We investigate thermodynamics of feedback processes driven by measurement. Regarding system and memory device as a composite system, mutual information as a measure of correlation between the two constituents contributes to the entropy of…

Actively inferring user preferences, for example by asking good questions, is important for any human-facing decision-making system. Active inference allows such systems to adapt and personalize themselves to nuanced individual preferences.…

Predictive statistical mechanics is a form of inference from available data, without additional assumptions, for predicting reproducible phenomena. By applying it to systems with Hamiltonian dynamics, a problem of predicting the macroscopic…

As powerful as machine learning (ML) techniques are in solving problems involving data with large dimensionality, explaining the results from the fitted parameters remains a challenging task of utmost importance, especially in physics…

Monitoring the magnet temperature in permanent magnet synchronous motors (PMSMs) for automotive applications is a challenging task for several decades now, as signal injection or sensor-based methods still prove unfeasible in a commercial…

From the Horowitz-Esposito stochastic thermodynamical description of information flows in dynamical systems [J. M. Horowitz and M. Esposito, Phys. Rev. X4, 031015 (2014)], it is known that while the second law of thermodynamics is satisfied…

We consider stochastic thermodynamics as a theory of statistical inference for experimentally observed fluctuating time-series. To that end, we introduce a general framework for quantifying the knowledge about the dynamical state of the…

Stochastic thermodynamics as reviewed here systematically provides a framework for extending the notions of classical thermodynamics like work, heat and entropy production to the level of individual trajectories of well-defined…

In a thermodynamic process with measurement and feedback, the second law of thermodynamics is no longer valid. In its place, various second-law-like inequalities have been advanced that each incorporate a distinct additional term accounting…

Scientific discovery can be framed as a thermodynamic process in which an agent invests physical work to acquire information about an environment under a finite work budget. Using established results about the thermodynamics of computing,…

We demonstrate that Shannon's information entropy and the thermodynamic entropy of Boltzmann and Gibbs are quantitatively equivalent for real condensed-matter systems. By interpreting atomic configurations as information sources, we compute…