Related papers: Learning rate matrix and information-thermodynamic…
How do organisms recognize their environment by acquiring knowledge about the world, and what actions do they take based on this knowledge? This article examines hypotheses about organisms' adaptation to the environment from machine…
A basic task of information processing is information transfer (flow). Here we study a pair of Brownian particles each coupled to a thermal bath at temperature $T_1$ and $T_2$, respectively. The information flow in such a system is defined…
We have formulated a family of machine learning problems as the time evolution of Parametric Probabilistic Models (PPMs), inherently rendering a thermodynamic process. Our primary motivation is to leverage the rich toolbox of thermodynamics…
Learning systems acquire structured internal representations from data, yet classical information-theoretic results state that deterministic transformations do not increase information. This raises a fundamental question: how can learning…
We consider the thermodynamic properties of systems in contact with an information source and focus on the consequences of energetic cost associated with the exchange of information. To this end we introduce the model of a thermal tape and…
Biological systems have to build models from their sensory data that allow them to efficiently process previously unseen inputs. Here, we study a neural network learning a linearly separable rule using examples provided by a teacher. We…
Using information theory we derive a thermodynamics for systems evolving under a collective motion, i.e. under a time-odd constraint. An illustration within the Lattice gas Model is given for two model cases: a collision between two complex…
We establish universal relations between pattern formation and dissipation with a geometric approach to nonequilibrium thermodynamics of deterministic reaction-diffusion systems. We first provide a way to systematically decompose the…
In the Internet of Things, learning is one of most prominent tasks. In this paper, we consider an Internet of Things scenario where federated learning is used with simultaneous transmission of model data and wireless power. We investigate…
Adaptive physical and biological systems continually process fluctuating information from their environments. When the environment is nonstationary, inference itself becomes a nonequilibrium process with thermodynamic cost. We analyse a…
In this work, we investigate the relation between the concept of ``information rate'', an information geometric method for measuring the speed of the time evolution of the statistical states of a stochastic process, and stochastic…
Adaptive systems -- such as a biological organism gaining survival advantage, an autonomous robot executing a functional task, or a motor protein transporting intracellular nutrients -- must model the regularities and stochasticity in their…
In thermal environments, information processing requires thermodynamic costs determined by the second law of thermodynamics. Information processing within finite time is particularly important, since fast information processing has…
We study the thermodynamic properties induced by non-reciprocal interactions between stochastic degrees of freedom in time- and space-continuous systems. We show that, under fairly general conditions, non-reciprocal coupling alone implies a…
We provide a unified thermodynamic formalism describing information transfers in autonomous as well as nonautonomous systems described by stochastic thermodynamics. We demonstrate how information is continuously generated in an auxiliary…
Recent experiments have indicated that many biological systems self-organise near their critical point, which hints at a common design principle. While it has been suggested that information transmission is optimized near the critical…
An information-theoretic framework is introduced to analyze last-layer embedding, focusing on learned representations for regression tasks. We define representation-rate and derive limits on the reliability with which input-output…
Information is instrumental in our understanding of thermodynamics. Their interplay has been studied through completely degenerate Hamiltonians whereby the informational contributions to thermodynamic transformations can be isolated. In…
This paper proposes a novel perspective on learning, positing it as the pursuit of dynamical invariants -- data combinations that remain constant or exhibit minimal change over time as a system evolves. This concept is underpinned by both…
We show that the mutual information between the representation of a learning machine and the hidden features that it extracts from data is bounded from below by the relevance, which is the entropy of the model's energy distribution. Models…