Related papers: On the form of the relative entropy between measur…
We introduce the notion of relative volume entropy for two spacetimes with preferred compact spacelike foliations. This is accomplished by applying the notion of Kullback-Leibler divergence to the volume elements induced on spacelike…
The Kullback-Leibler divergence or relative entropy is an information-theoretic measure between statistical models that play an important role in measuring a distance between random variables. In the study of complex systems, random fields…
This work explores connections between the quantum relative entropy of two faithful states $\rho,\sigma$ (i.e. full-rank density matrices) and the Kullback-Leibler divergences of classical measures $\mu,\nu$. Here, $\mu$ and $\nu$ are…
In the context of averaging an inhomogeneous cosmological model, we propose a natural measure identical to the Kullback-Leibler relative information entropy, which expresses the distinguishability of the local inhomogeneous density field…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
Many thermodynamic relations involve inequalities, with equality if a process does not involve dissipation. In this article we provide equalities in which the dissipative contribution is shown to involve the relative entropy (a.k.a.…
A classic definition of multisensory integration (MI) has been proposed as ``the presence of a (statistically) significant change in the response to a cross-modal stimulus complex compared to unimodal stimuli''. However, this general…
In a variety of applications it is important to extract information from a probability measure $\mu$ on an infinite dimensional space. Examples include the Bayesian approach to inverse problems and possibly conditioned) continuous time…
We give a new characterization of relative entropy, also known as the Kullback-Leibler divergence. We use a number of interesting categories related to probability theory. In particular, we consider a category FinStat where an object is a…
For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one…
The necessary information to distinguish a local inhomogeneous mass density field from its spatial average on a compact domain of the universe can be measured by relative information entropy. The Kullback-Leibler (KL) formula arises very…
We establish convergence in the diffusive limit from entropy weak solutions of the equations of compressible gas dynamics with friction to the porous media equation away from vacuum. The result is based on a Lyapunov type of functional…
Many thermodynamic relations involve inequalities, with equality if a process does not involve dissipation. In this article we provide equalities in which the dissipative contribution is shown to involve the relative entropy (a.k.a.…
Relative entropy is a fundamental class of distances between probability distributions, with widespread applications in probability theory, statistics, and machine learning. In this work, we study relative entropy from a categorical…
Trajectory Inference (TI) seeks to recover latent dynamical processes from snapshot data, where only independent samples from time-indexed marginals are observed. In applications such as single-cell genomics, destructive measurements make…
In this paper we review various information-theoretic characterizations of the approach to equilibrium in biological systems. The replicator equation, evolutionary game theory, Markov processes and chemical reaction networks all describe…
For a measurable map $T$ and a sequence of $T$-invariant probability measures $\mu_n$ that converges in some sense to a $T$-invariant probability measure $\mu$, an estimate from below for the Kolmogorov--Sinai entropy of $T$ with respect to…
The limits of scaled relative entropies between probability distributions associated with N-particle weakly interacting Markov processes are considered. The convergence of such scaled relative entropies is established in various settings.…
Heisenberg's uncertainty principle has recently led to general measurement uncertainty relations for quantum systems: incompatible observables can be measured jointly or in sequence only with some unavoidable approximation, which can be…
We introduce a new information-theoretic formulation of quantum measurement uncertainty relations, based on the notion of relative entropy between measurement probabilities. In the case of a finite-dimensional system and for any approximate…