Related papers: Knowledge as a Breaking of Ergodicity
It is shown that the dual to the linear programming problem that arises in constraint-based models of metabolism can be given a thermodynamic interpretation in which the shadow prices are chemical potential analogues, and the objective is…
Generative models face a fundamental challenge: they must simultaneously learn high-level semantic concepts (what to generate) and low-level synthesis details (how to generate it). Conventional end-to-end training entangles these distinct,…
Neural language models can be successfully trained on source code, leading to applications such as code completion. However, their versatile autoregressive self-supervision objective overlooks important global sequence-level features that…
Developing high-precision models of the nuclear force and propagating the associated uncertainties in quantum many-body calculations of nuclei and nuclear matter remain key challenges for ab initio nuclear theory. In the present work we…
The optimization of the conversion of thermal energy into work and the minimization of dissipation for nano- and mesoscopic systems is a complex challenge because of the important role fluctuations play on the dynamics of small systems. We…
This work studies the learning problem of the energy-based prior model and the multi-layer generator model. The multi-layer generator model, which contains multiple layers of latent variables organized in a top-down hierarchical structure,…
The quantum simulation of gauge theories on synthetic quantum matter devices has gained a lot of traction in the last decade, making possible the observation of a range of exotic quantum many-body phenomena. In this work, we consider the…
Civilization produces knowledge, which acts as the driving force of its development. A macro-model of civilization that accounts for the effect of knowledge production on population, energy consumption and environmental conditions is…
The concept of weak ergodicity breaking is defined and studied in the context of deterministic dynamics. We show that weak ergodicity breaking describes a weakly chaotic dynamical system: a nonlinear map which generates subdiffusion…
The study of intelligent systems explains behaviour in terms of economic rationality. This results in an optimization principle involving a function or utility, which states that the system will evolve until the configuration of maximum…
We develop a thermodynamic theory for machine learning (ML) systems. Similar to physical thermodynamic systems which are characterized by energy and entropy, ML systems possess these characteristics as well. This comparison inspire us to…
We study the breaking of ergodicity measured in terms of return probability in the evolution of a quantum state of a spin chain. In the non ergodic phase a quantum state evolves in a much smaller fraction of the Hilbert space than would be…
Dynamic nonlinear systems exhibit distortions arising from coupled static and dynamic effects. Their intertwined nature poses major challenges for data-driven modeling. This paper presents a theoretical framework grounded in structured…
Estimating the free energy, as well as other thermodynamic observables, is a key task in lattice field theories. Recently, it has been pointed out that deep generative models can be used in this context [1]. Crucially, these models allow…
How do cells tune emergent properties at the scale of tissues? One class of such emergent behaviors are rigidity transitions, in which a tissue changes from a solid-like to a fluid-like state or vice versa. Here, we introduce a new way for…
We develop a thermodynamic theory of algorithmic catalysis within the watts-per-intelligence framework, identifying reusable computational structures that reduce irreversible operations for a task class while satisfying bounded restoration…
Perfectly rational decision-makers maximize expected utility, but crucially ignore the resource costs incurred when determining optimal actions. Here we propose an information-theoretic formalization of bounded rational decision-making…
Many complex systems satisfy a set of constraints on their degrees of freedom, and at the same time, they are able to work and adapt to different conditions. Here, we describe the emergence of this ability in a simplified model in which the…
The degrees of freedom of multi-compartment mathematical models for energy metabolism of a neuron-astrocyte complex may offer a key to understand the different ways in which the energetic needs of the brain are met. In this paper we address…
Energy-based models (EBMs) are a simple yet powerful framework for generative modeling. They are based on a trainable energy function which defines an associated Gibbs measure, and they can be trained and sampled from via well-established…