Related papers: Kullback-Leibler Potential for Non-Ergodic Replica…

Mutual Information (MI) is a fundamental measure of statistical dependence widely used in representation learning. While direct optimization of MI via its definition as a Kullback-Leibler divergence (KLD) is often intractable, many recent…

The second law of thermodynamics is probabilistic in nature. Its formulation requires that the state of a system be described by a probability distribution. A natural question, thereby, arises as to whether a prior knowledge about the state…

Landauer's principle sets fundamental thermodynamical constraints for classical and quantum information processing, thus affecting not only various branches of physics, but also of computer science and engineering. Despite its importance,…

Landauer's "principle" claims that erasing one bit of information necessarily dissipates at least Tln2 of heat into the surroundings, making a possibly logically irreversible operation also thermodynamically irreversible. It is commonly…

The delocalization or scrambling of quantum information has emerged as a central ingredient in the understanding of thermalization in isolated quantum many-body systems. Recently, significant progress has been made analytically by modeling…

Information theory is an outstanding framework to measure uncertainty, dependence and relevance in data and systems. It has several desirable properties for real world applications: it naturally deals with multivariate data, it can handle…

Information has an entropic character which can be analyzed within the Statistical Theory in molecular systems. R. Landauer and C.H. Bennett showed that a logical copy can be carried out in the limit of no dissipation if the computation is…

Thermodynamics establishes that information acquired through measurement can be converted into work, as exemplified by Maxwell's demon and Szilard engines. Most experimental realizations of information engines, however, implicitly assume…

The Kullback-Leibler (KL) divergence is a fundamental equation of information theory that quantifies the proximity of two probability distributions. Although difficult to understand by examining the equation, an intuition and understanding…

In this article, we focus on erasure of a bit of information in finite time. Landauer's principle states that the average heat dissipation due to erasure of information is k_B T ln 2, which is achievable only in an asymptotic manner. Recent…

The reversible computation paradigm aims to provide a new foundation for general classical digital computing that is capable of circumventing the thermodynamic limits to the energy efficiency of the conventional, non-reversible digital…

As early as 1867, two years after the introduction of the concept of entropy by Clausius, Maxwell showed that the limitations imposed by the second law of thermodynamics depend on the information that one possesses about the state of a…

Maximizing the Kullback-Leibler divergence (KLD) is a fundamental problem in waveform design for active sensing and hypothesis testing, as it directly relates to the error exponent of detection probability. However, the associated…

A well-known technique in estimating probabilities of rare events in general and in information theory in particular (used, e.g., in the sphere-packing bound), is that of finding a reference probability measure under which the event of…

The Landauer principle states that any logically irreversible information processing must be accompanied by dissipation into the environment. In this study, we investigate the heat dissipation associated with finite-time information erasure…

We investigate the link between information and thermodynamics embodied by Landauer's principle in the open dynamics of a multipartite quantum system. Such irreversible dynamics is described in terms of a collisional model with a finite…

The irreversible entropy increase described by the second law of thermodynamics is fundamentally tied to thermalization and the emergence of equilibrium. In the first part of our work (Ref: arXiv.2503.04152), we constructed an isolated gas…

Since its introduction, the partial information decomposition (PID) has emerged as a powerful, information-theoretic technique useful for studying the structure of (potentially higher-order) interactions in complex systems. Despite its…

This letter provides an adaptive resampling method. It determines the number of particles to resample so that the Kullback-Leibler distance (KLD) between distribution of particles before resampling and after resampling does not exceed a…

We show a relationship between the entropy production in stochastic thermodynamics and the stochastic interaction in the information integrated theory. To clarify this relationship, we newly introduce an information geometric interpretation…