Related papers: Thermodynamic Irreversibility of Training Algorith…
The reversibility and recurrence paradoxes are key issues that have been left unsolved in researches on the foundation of thermodynamics since the 19th century. This article shows that (1) the reversibility paradox can be overcome if we pay…
Optimal (reversible) processes in thermodynamics can be modelled as step-by-step processes, where the system is successively thermalized with respect to different Hamiltonians by an external thermal bath. However, in practice interactions…
In this article, we analyze three classes of time-reversal of a Markov process with Gaussian noise on a manifold. We first unveil a commutativity constraint for the most general of these time-reversals to be well defined. Then we give a…
In addition to the Riemannian metricization of the thermodynamic state space, local relaxation times offer a natural time scale, too. Generalizing existing proposals, we relate {\it thermodynamic} time scale to the standard kinetic…
Recent AI systems compress the distance between capability growth and capability deployment. Earlier high-risk technologies were slowed by capital intensity, physical bottlenecks, organizational inertia, and specialized supply chains. By…
Empirical risk minimization is a standard principle for choosing algorithms in learning theory. In this paper we study the properties of empirical risk minimization for time series. The analysis is carried out in a general framework that…
Many Artificial Intelligence (AI) algorithms are inspired by physics and employ stochastic fluctuations. We connect these physics-inspired AI algorithms by unifying them under a single mathematical framework that we call Thermodynamic AI.…
Background: Many published machine learning studies are irreproducible. Issues with methodology and not properly accounting for variation introduced by the algorithm themselves or their implementations are attributed as the main…
The Turing machine halting problem can be explained by several factors, including arithmetic logic irreversibility and memory erasure, which contribute to computational uncertainty due to information loss during computation. Essentially,…
Learning governing dynamics from data is a common goal across the sciences, yet it is only well-posed when the underlying mechanisms are identifiable. In practice, many data-driven methods implicitly assume identifiability; when this…
Within the general formalism of quantum theory irreversibility and the arrow of time in the evolution of various physical systems are studied. Irreversible behavior often manifests itself in the guise of entropy production. This motivates…
We analyze underdamped Brownian motion in non-isothermal media with quadratic, linear, and piecewise-constant temperature profiles. Exact identities for entropy production and entropy extraction are derived, addressing whether a vanishing…
We attempt to characterize irreversibility of a dynamical system from the existence of different forward and backward mathematical representations depending on the direction of the time arrow. Such different representations have been…
A unified thermodynamic framework for characterization of functional materials is developed. This framework encompasses linear reversible and irreversible processes with thermal, electrical, magnetic, and/or mechanical effects coupled. The…
The depth of networks plays a crucial role in the effectiveness of deep learning. However, the memory requirement for backpropagation scales linearly with the number of layers, which leads to memory bottlenecks during training. Moreover,…
Temporal difference (TD) learning is a fundamental algorithm for estimating value functions in reinforcement learning. Recent finite-time analyses of TD with linear function approximation quantify its theoretical convergence rate. However,…
We discuss the thermodynamics of closed quantum systems driven out of equilibrium by a change in a control parameter and undergoing a unitary process. We compare the work actually done on the system with the one that would be performed…
Flocking in $d=2$ is a genuine non-equilibrium phenomenon for which irreversibility is an essential ingredient. We study a class of minimal flocking models whose only source of irreversibility is self-propulsion and use the entropy…
Machine unlearning, an emerging research topic focusing on compliance with data privacy regulations, enables trained models to remove the information learned from specific data. While many existing methods indirectly address this issue by…
Reversibility in artificial neural networks allows us to retrieve the input given an output. We present feature alignment, a method for approximating reversibility in arbitrary neural networks. We train a network by minimizing the distance…