Related papers: Kullback-Leibler Potential for Non-Ergodic Replica…
We study thermodynamic limits when controllers operate with only partial observability of internal correlations in multipartite systems. Understanding the costs imposed by lack of information is crucial in settings where agents must act…
Informational contributions to thermodynamics can be studied in isolation by considering systems with fully-degenerate Hamiltonians. In this regime, being in non-equilibrium -- termed informational non-equilibrium -- provides thermodynamic…
Experimental designs are tools which can drastically reduce the number of simulations required by time-consuming computer codes. One strategy for selecting the values of the inputs, whose response is to be observed, is to choose these…
The energy cost of erasing a bit of information was fundamentally lower bounded by Landauer, in terms of the temperature of its environment: $W\geq k_\mathrm{B} T \ln 2$. However, in real electronic devices, the information-bearing system…
In this paper, we delve deeper into the Kullback-Leibler (KL) Divergence loss and mathematically prove that it is equivalent to the Decoupled Kullback-Leibler (DKL) Divergence loss that consists of 1) a weighted Mean Square Error (wMSE)…
The Kullback-Leibler divergence or relative entropy is an information-theoretic measure between statistical models that play an important role in measuring a distance between random variables. In the study of complex systems, random fields…
The deepening penetration of renewable resources into power systems entails great difficulties that have not been surmounted satisfactorily. An issue that merits special attention is the short-term planning of power systems under net load…
We investigate thermodynamics of feedback processes driven by measurement. Regarding system and memory device as a composite system, mutual information as a measure of correlation between the two constituents contributes to the entropy of…
The deep connection between entropy and information is discussed in terms of both classical and quantum physics. The mechanism of information transfer between systems via entanglement is explored in the context of decoherence theory. The…
We derive independence tests by means of dependence measures thresholding in a semiparametric context. Precisely, estimates of phi-mutual informations, associated to phi-divergences between a joint distribution and the product distribution…
Knowledge distillation involves transferring knowledge from large, cumbersome teacher models to more compact student models. The standard approach minimizes the Kullback-Leibler (KL) divergence between the probabilistic outputs of a teacher…
This is a study of the information evolution of complex systems by geometrical consideration. We look at chaotic systems evolving in fractal phase space. The entropy change in time due to the fractal geometry is assimilated to the…
We present a critical examination of the difficulties with the quantum versions of a lifted weight that are widely used as work storage systems in quantum thermodynamics. To overcome those difficulties, we turn to the strong connections…
Physical systems are often simulated using a stochastic computation where different final states result from identical initial states. Here, we derive the minimum energy cost of simulating a complex data set of a general physical system…
Time-delay embedding is a powerful technique for reconstructing the state space of nonlinear time series. However, the fidelity of reconstruction relies on the assumption that the time-delay map is an embedding, which is implicitly…
A known aspect of the Clausius inequality is that an equilibrium system subjected to a squeezing $\d S<0$ of its entropy must release at least an amount $|\dbarrm Q|=T|\d S|$ of heat. This serves as a basis for the Landauer principle, which…
Here, we investigate the uncertainty of dynamical observables in classical systems manipulated by repeated measurements and feedback control; the precision should be enhanced in the presence of an external controller but limited by the…
Biological sensory systems react to changes in their surroundings. They are characterized by fast response and slow adaptation to varying environmental cues. Insofar as sensory adaptive systems map environmental changes to changes of their…
We integrate information-theoretic concepts into the design and analysis of optimistic algorithms and Thompson sampling. By making a connection between information-theoretic quantities and confidence bounds, we obtain results that relate…
Information processing at the molecular scale is limited by thermal fluctuations. This can cause undesired consequences in copying information since thermal noise can lead to errors that can compromise the functionality of the copy. For…