Related papers: Thermodynamic Irreversibility of Training Algorith…

When learning behavior, training data is often generated by the learner itself; this can result in unstable training dynamics, and this problem has particularly important applications in safety-sensitive real-world control tasks such as…

There is intense effort into understanding the universal properties of finite-time models of thermal machines---at optimal performance---such as efficiency at maximum power, coefficient of performance at maximum cooling power, and other…

Temporal difference (TD) learning is a foundational algorithm in reinforcement learning (RL). For nearly forty years, TD learning has served as a workhorse for applied RL as well as a building block for more complex and specialized…

The 2nd law of thermodynamics yields an irreversible increase in entropy until thermal equilibrium is achieved. This irreversible increase is often assumed to require large and complex systems to emerge from the reversible microscopic laws…

Experimental data bases are typically very large and high dimensional. To learn from them requires to recognize important features (a pattern), often present at scales different to that of the recorded data. Following the experience…

The literature on dynamical systems has, for the most part, considered self-oscillators (i.e., systems capable of generating and maintaining a periodic motion at the expense of an external energy source with no corresponding periodicity)…

We study dynamical reversibility in stationary stochastic processes from an information theoretic perspective. Extending earlier work on the reversibility of Markov chains, we focus on finitary processes with arbitrarily long conditional…

Unlearning in large language models (LLMs) aims to remove specified data, but its efficacy is typically assessed with task-level metrics like accuracy and perplexity. We show that these metrics can be misleading, as models can appear to…

A dynamical system is said to be reversible if, given an output, the input can always be recovered in a well-posed manner. Nevertheless, we argue that reversible systems that have a time-reversal symmetry, such as the Nonlinear…

Time irreversibility, defined as the lack of invariance of the statistical properties of a system or time series under the operation of time reversal, has received an increasing attention during the last decades, thanks to the information…

Quantifying irreversibility of a system using finite information constitutes a major challenge in stochastic thermodynamics. We introduce an observable that measures the time-reversal asymmetry between two states after a given time lag. Our…

The relation between time series irreversibility and entropy production has been recently investigated in thermodynamic systems operating away from equilibrium. In this work we explore this concept in the context of financial time series.…

Irreversible processes accomplished in a fixed time involve nonlinearly coupled flows of matter, energy, and information. Here, using entropy production as an example, we show how thermodynamic uncertainty relations and speed limits on…

In many physical systems, inputs related by intrinsic system symmetries are mapped to the same output. When inverting such systems, i.e., solving the associated inverse problems, there is no unique solution. This causes fundamental…

The aim of this paper is to determine lost works in a molecular engine and compare results with macro (classical) heat engines. Firstly, irreversible thermodynamics are reviewed for macro and molecular cycles. Secondly, irreversible…

As a means to balance the growth of the AI industry with the need for privacy protection, machine unlearning plays a crucial role in realizing the ``right to be forgotten'' in artificial intelligence. This technique enables AI systems to…

Current research in statistical mechanics mostly concerns the investigation of out-of-equilibrium, irreversible processes, which are ubiquitous in nature and still far from being theoretically understood. Even the precise characterization…

The dynamics of time-reversible systems are statistically indistinguishable when observed forward or backward in time. A rich literature of statistical methods to distinguish irreversible dynamics from the reversible dynamics of linear,…

We propose that unconstrained artificial intelligence obeys a Second Law analogous to thermodynamics, where ethical entropy, defined as a measure of divergence from intended goals, increases spontaneously without continuous alignment work.…

With the rapid discovery of emergent phenomena in deep learning and large language models, understanding their cause has become an urgent need. Here, we propose a rigorous entropic-force theory for understanding the learning dynamics of…