Related papers: Stochastic Thermodynamics of Associative Memory
Stochastic thermodynamics is a framework for describing non-equilibrium processes at the level of fluctuating trajectories, where the state of a system evolves as a stochastic time series, allowing thermodynamic quantities such as work,…
The relationship between the thermodynamic and computational characteristics of dynamical physical systems has been a major theoretical interest since at least the 19th century, and has been of increasing practical importance as the…
The rapid advancement of models based on artificial intelligence demands innovative monitoring techniques which can operate in real time with low computational costs. In machine learning, especially if we consider artificial neural networks…
A unified thermodynamic formalism describing the efficiency of learning is proposed. First, we derive an inequality, which is more strength than Clausius's inequality, revealing the lower bound of the entropy-production rate of a subsystem.…
Dense Associative Memory (DAM) models have been attracting renewed attention since they were shown to be robust to adversarial examples and closely related to cutting edge machine learning paradigms, such as the attention mechanism and…
We study thermodynamic processes in contact with a heat bath that may have an arbitrary time-varying periodic temperature profile. Within the framework of stochastic thermodynamics, and for models of thermo-dynamic engines in the idealized…
Modern Hopfield Neural Networks (HNNs), also known as Dense Associative Memories (DAMs), enhance the performance of simple recurrent neural networks by leveraging the nonlinearities in their energy functions. They have broad applications in…
In this study, we advance the understanding of non-equilibrium systems by deriving thermodynamic relations for a heat engine operating under an exponentially decreasing temperature profile. Such thermal configurations closely mimic…
We develop a statistical mechanical interpretation of algorithmic information theory by introducing the notion of thermodynamic quantities, such as free energy, energy, statistical mechanical entropy, and specific heat, into algorithmic…
A statistical mechanical distance constraint model (DCM) is presented that explicitly accounts for network rigidity among constraints present within a system. Constraints are characterized by local microscopic free energy functions.…
Recent advances at the intersection of control theory, neuroscience, and machine learning have revealed novel mechanisms by which dynamical systems perform computation. These advances encompass a wide range of conceptual, mathematical, and…
The increasing computational demands of modern AI systems have exposed fundamental limitations of digital hardware, driving interest in alternative paradigms for efficient large-scale inference. Dense Associative Memory (DenseAM) is a…
Dense Associative Memory (DAM) models generalize the classical Hopfield model by incorporating n-body or exponential interactions that greatly enhance storage capacity. While the criticality of DAM models has been largely investigated,…
Thermodynamics of nanoscale devices is an active area of research. Despite their noisy surrounding they often produce mechanical work (e.g. micro-heat engines), display rectified Brownian motion (e.g. molecular motors). This invokes…
Adaptive physical and biological systems continually process fluctuating information from their environments. When the environment is nonstationary, inference itself becomes a nonequilibrium process with thermodynamic cost. We analyse a…
Many Artificial Intelligence (AI) algorithms are inspired by physics and employ stochastic fluctuations. We connect these physics-inspired AI algorithms by unifying them under a single mathematical framework that we call Thermodynamic AI.…
Energy costs of information processing are growing exponentially. Bit erasure is a key problem in this energy-information nexus, and a number of seminal relationships have been deduced regarding the relationship between thermodynamic costs…
Dense Associative Memories or Modern Hopfield Networks have many appealing properties of associative memory. They can do pattern completion, store a large number of memories, and can be described using a recurrent neural network with a…
We set up a framework for quantum stochastic thermodynamics based solely on experimentally controllable, but otherwise arbitrary interventions at discrete times. Using standard assumptions about the system-bath dynamics and insights from…
Associative memory models such as the Hopfield network and its dense generalizations with higher-order interactions exhibit a "blackout catastrophe" -- a discontinuous transition where stable memory states abruptly vanish when the number of…