Related papers: Relative entropy via non-sequential recursive pair…
The necessary information to distinguish a local inhomogeneous mass density field from its spatial average on a compact domain of the universe can be measured by relative information entropy. The Kullback-Leibler (KL) formula arises very…
Relative entropy is a measure of distinguishability for quantum states, and plays a central role in quantum information theory. The family of Renyi entropies generalizes to Renyi relative entropies that include as special cases most entropy…
Quantum uncertainty relations are formulated in terms of relative entropy between distributions of measurement outcomes and suitable reference distributions with maximum entropy. This type of entropic uncertainty relation can be applied…
We proposed a learning algorithm for nonparametric estimation and on-line prediction for general stationary ergodic sources. We prepare histograms each of which estimates the probability as a finite distribution, and mixture them with…
In this paper, we derive exponential ergodicity in relative entropy for general kinetic SDEs under a partially dissipative condition. It covers non-equilibrium situations where the forces are not of gradient type and the invariant measure…
We establish convergence in the diffusive limit from entropy weak solutions of the equations of compressible gas dynamics with friction to the porous media equation away from vacuum. The result is based on a Lyapunov type of functional…
A plug-in estimator of entropy is the entropy of the distribution where probabilities of symbols or blocks have been replaced with their relative frequencies in the sample. Consistency and asymptotic unbiasedness of the plug-in estimator…
The irreversible currents and entropy production rate of a dilute colloidal suspension are calculated using the linear irreversible thermodynamics and the linear response theory. The \anomalous" or \hidden" entropy recently discussed in the…
The maximum entropy formalism developed by Jaynes determines the relevant ensemble in nonequilibrium statistical mechanics by maximising the entropy functional subject to the constraints imposed by the available information. We present an…
Model or variable selection is usually achieved through ranking models according to the increasing order of preference. One of methods is applying Kullback-Leibler distance or relative entropy as a selection criterion. Yet that will raise…
It is supposed that the exponential multiplier in the method of the non-equilibrium statistical operator (Zubarev`s approach) can be considered as a distribution density of the past lifetime of the system, and can be replaced by an…
Here, we propose a new tool to estimate the complexity of a time series: the entropy of difference (ED). The method is based solely on the sign of the difference between neighboring values in a time series. This makes it possible to…
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…
We investigate the analogy between the renormalization group (RG) and deep neural networks, wherein subsequent layers of neurons are analogous to successive steps along the RG. In particular, we quantify the flow of information by…
We study the fluctuations of ergodic sums using global and local specifications on periodic points. We obtain Lindeberg-type central limit theorems in both situations. As an application, when the system possesses a unique measure of maximal…
Relative entropy serves as a cornerstone concept in quantum information theory. In this work, we study relative entropy of random states from major generic state models of Hilbert-Schmidt and Bures-Hall ensembles. In particular, we derive…
A geometrical framework for the definition of entropy in General Relativity via Noether theorem is briefly recalled and the entropy of Taub-Bolt Euclidean solutions of Einstein equations is then obtained as an application. The computed…
This archiving article consists of several short reports on the discussions between the two authors over the past two years at Oxford and Madrid, and their work carried out during that period on the upper bound of the Kullback-Leibler…
A lower bound of the reduced relative entropy is given by the use of a variational expression. The reduced Tsallis relative entropy is defined and some results are given. In particular, the convexity of the reduced Tsallis relative entropy…
By establishing Multiplicative Ergodic Theorem for commutative transformations on a separable infinite dimensional Hilbert space, in this paper, we investigate Pesin's entropy formula and SRB measures of a finitely generated random…