Related papers: Knowledge as a Breaking of Ergodicity
We use a formal correspondence between thermodynamics and inference, where the number of samples can be thought of as the inverse temperature, to study a quantity called ``learning capacity'' which is a measure of the effective…
At low temperatures the configurational phase space of a macroscopic complex system (e.g., a spin-glass) of $N\sim 10^{23}$ interacting particles may split into an exponential number $\Omega_s \sim \exp({\rm const} \times N)$ of ergodic…
The behavior of lattice models in which time reversibility is enforced at the level of trajectories (microscopic reversibility) is studied analytically. Conditions for ergodicity breaking are explored, and a few examples of systems…
We investigate one-dimensional driven diffusive systems where particles may also be created and annihilated in the bulk with sufficiently small rate. In an open geometry, i.e., coupled to particle reservoirs at the two ends, these systems…
In the vicinity of a phase transition ergodicity can be broken. Here, different initial many-body configurations evolve towards one of several fixed points, which are macroscopically distinguishable through an order parameter. This…
We study the nature and mechanisms of broken ergodicity (BE) in specific random walk models corresponding to diffusion on random potential surfaces, in both one and high dimension. Using both rigorous results and nonrigorous methods, we…
The second law of thermodynamics uses change in free energy of macroscopic systems to set a bound on performed work. Ergotropy plays a similar role in microscopic scenarios, and is defined as the maximum amount of energy that can be…
Generative neural networks can produce data samples according to the statistical properties of their training distribution. This feature can be used to test modern computational neuroscience hypotheses suggesting that spontaneous brain…
Energy-based learning is a powerful learning paradigm that encapsulates various discriminative and generative approaches. An energy-based model (EBM) is typically formed of inner-model(s) that learn a combination of the different features…
We introduce a variant of the asymmetric random average process with continuous state variables where the maximal transport is restricted by a cutoff. For periodic boundary conditions, we show the existence of a phase transition between a…
How many parameters are required for a model to execute a given task? It has been argued that large language models, pre-trained via self-supervised learning, exhibit emergent capabilities such as multi-step reasoning as their number of…
Learning systems acquire structured internal representations from data, yet classical information-theoretic results state that deterministic transformations do not increase information. This raises a fundamental question: how can learning…
RNA function crucially depends on its structure. Thermodynamic models currently used for secondary structure prediction rely on computing the partition function of folding ensembles, and can thus estimate minimum free-energy structures and…
The potential benefits of model-free reinforcement learning to real robotics systems are limited by its uninformed exploration that leads to slow convergence, lack of data-efficiency, and unnecessary interactions with the environment. To…
A unified thermodynamic formalism describing the efficiency of learning is proposed. First, we derive an inequality, which is more strength than Clausius's inequality, revealing the lower bound of the entropy-production rate of a subsystem.…
A thermodynamic formalism describing the efficiency of information learning is proposed, which is applicable for stochastic thermodynamic systems with multiple internal degree of freedom. The learning rate, entropy production rate (EPR),…
Thermodynamics teaches that if a system initially off-equilibrium is coupled to work sources, the maximum work that it may yield is governed by its energy and entropy. For finite systems this bound is usually not reachable. The maximum…
We present a differentiable formalism for learning free energies that is capable of capturing arbitrarily complex model dependencies on coarse-grained coordinates and finite-temperature response to variation of general system parameters.…
From a dynamical viewpoint, basic phase transitions of statistical mechanics can be regarded as a breaking of ergodicity. While many random models exhibiting such transitions at the thermodynamics limit exist, finite-dimensional examples…
In many real-world situations, there are constraints on the ways in which a physical system can be manipulated. We investigate the entropy production (EP) and extractable work involved in bringing a system from some initial distribution $p$…