Related papers: On Relations Between the Relative entropy and $\ch…
General hyperbolic systems of balance laws with inhomogeneity in space and time in all constitutive functions are studied in the context of relative entropy. A framework is developed in this setting that contributes to a measure-valued weak…
Relating thermodynamic and kinetic properties is a conceptual challenge with many practical benefits. Here, based on first principles, we derive a rigorous inequality relating the entropy and the dynamic propagator of particle…
We derive a relative entropy inequality for capillary compressible fluids with density dependent viscosity. Applications in the context of weak-strong uniqueness analysis, pressureless fluids and high-Mach number flows are presented.
Distance correlation has gained much recent attention in the data science community: the sample statistic is straightforward to compute and asymptotically equals zero if and only if independence, making it an ideal choice to discover any…
Divergence functions are measures of distance or dissimilarity between probability distributions that serve various purposes in statistics and applications. We propose decompositions of Wasserstein and Cram\'er distances$-$which compare two…
The theory of large deviations deals with the probabilities of rare events (or fluctuations) that are exponentially small as a function of some parameter, e.g., the number of random components of a system, the time over which a stochastic…
We show that the natural scaling of measurement for a particular problem defines the most likely probability distribution of observations taken from that measurement scale. Our approach extends the method of maximum entropy to use…
This paper deals with the diffusive limit of the Jin and Xin model and its approximation by an asymptotic preserving finite volume scheme. At the continuous level, we determine a convergence rate to the diffusive limit by means of a…
Many thermodynamic relations involve inequalities, with equality if a process does not involve dissipation. In this article we provide equalities in which the dissipative contribution is shown to involve the relative entropy (a.k.a.…
Information theoretic measures (entropies, entropy rates, mutual information) are nowadays commonly used in statistical signal processing for real-world data analysis. The present work proposes the use of Auto Mutual Information (Mutual…
We study minimization of a parametric family of relative entropies, termed relative $\alpha$-entropies (denoted $\mathscr{I}_{\alpha}(P,Q)$). These arise as redundancies under mismatched compression when cumulants of compressed lengths are…
The asymptotic correspondence between the probability mass function of the $q$-deformed multinomial distribution and the $q$-generalised Kullback-Leibler divergence, also known as Tsallis relative entropy, is established. The probability…
We develop information-theoretic measures of spatial structure and pattern in more than one dimension. As is well known, the entropy density of a two-dimensional configuration can be efficiently and accurately estimated via a converging…
Shannon entropy is not the only entropy that is relevant to machine-learning datasets, nor possibly even the most important one. Traditional entropies such as Shannon entropy capture information represented by elements' frequencies but not…
Two families of dependence measures between random variables are introduced. They are based on the R\'enyi divergence of order $\alpha$ and the relative $\alpha$-entropy, respectively, and both dependence measures reduce to Shannon's mutual…
We introduce the notion of relative volume entropy for two spacetimes with preferred compact spacelike foliations. This is accomplished by applying the notion of Kullback-Leibler divergence to the volume elements induced on spacelike…
By using Araki's relative entropy, Lieb's convexity and the theory of singular integrals, we compute the mutual information associated with free fermions, and we deduce many results about entropies for chiral CFT's which are embedded into…
In this paper we discuss various connections between geometric discrepancy measures, such as discrepancy with respect to convex sets (and convex sets with smooth boundary in particular), and applications to numerical analysis and…
We review old and recent finite de Finetti theorems in total variation distance and in relative entropy, and we highlight their connections with bounds on the difference between sampling with and without replacement. We also establish two…
A new upper bound on the relative entropy is derived as a function of the total variation distance for probability measures defined on a common finite alphabet. The bound improves a previously reported bound by Csisz\'ar and Talata. It is…