Related papers: On the Kullback-Leibler divergence between pairwis…
This paper investigates the best known bounds on the quadratic Gaussian distortion-rate-perception function with limited common randomness for the Kullback-Leibler divergence-based perception measure, as well as their counterparts for the…
In this letter, we propose a novel statistical method to measure which system is better suited to probe small deviations from the usual quantum behavior. Such deviations are motivated by a number of theoretical and phenomenological…
Comparing probability distributions is an indispensable and ubiquitous task in machine learning and statistics. The most common way to compare a pair of Borel probability measures is to compute a metric between them, and by far the most…
General characterization of physical measurements is discussed within the framework of a classical information theory. Uncertainty relation for simultaneous measurements of two physical observables is defined in this framework for…
In this paper we shall consider one parametric generalization of some non-symmetric divergence measures. The \textit{non-symmetric divergence measures} are such as: Kullback-Leibler \textit{relative information}, $\chi…
This paper proposes two linear projection methods for supervised dimension reduction using only the first and second-order statistics. The methods, each catering to a different parameter regime, are derived under the general Gaussian model…
Obtaining an accurate estimate of the underlying covariance matrix from finite sample size data is challenging due to sample size noise. In recent years, sophisticated covariance-cleaning techniques based on random matrix theory have been…
We propose an information-geometric signature of nonconservative driving that detects violations of detailed balance using the Kullback--Leibler divergence and the Fisher information. For Markov jump processes satisfying detailed balance,…
We introduce a new transformation called \emph{relative differential-escort}, which extends the usual differential-escort transformation by relating the change of variable to a reference probability density. As an application of it, we…
The correlation distance quantifies the statistical independence of two classical or quantum systems, via the distance from their joint state to the product of the marginal states. Tight lower bounds are given for the mutual information…
The Kullback-Leibler (KL) divergence is not a proper distance metric and does not satisfy the triangle inequality, posing theoretical challenges in certain practical applications. Existing work has demonstrated that KL divergence between…
Estimating the Kullback-Leibler (KL) divergence between random variables is a fundamental problem in statistical analysis. For continuous random variables, traditional information-theoretic estimators scale poorly with dimension and/or…
Log-Euclidean distances are commonly used to quantify the similarity between positive definite matrices using geometric considerations. This paper analyzes the behavior of this distance when it is used to measure closeness between…
Relative entropy serves as a cornerstone concept in quantum information theory. In this work, we study relative entropy of random states from major generic state models of Hilbert-Schmidt and Bures-Hall ensembles. In particular, we derive…
We present a derivation of the Kullback Leibler (KL)-Divergence (also known as Relative Entropy) for the von Mises Fisher (VMF) Distribution in $d$-dimensions.
This document shows how to obtain the Jacobian and Hessian matrices of the Kullback-Leibler divergence between two multivariate Gaussian distributions, using the first and second-order differentials. The presented derivations are based on…
Consider the nonparametric logistic regression problem. In the logistic regression, we usually consider the maximum likelihood estimator, and the excess risk is the expectation of the Kullback-Leibler (KL) divergence between the true and…
We present a definition of the distance between probability distributions. Our definition is based on the $L_1$ norm on space of probability measures. We compare our distance with the well-known Kullback-Leibler divergence and with the…
Extreme Value Theory plays an important role to provide approximation results for the extremes of a sequence of independent random variables when their distribution is unknown. An important one is given by the {generalised Pareto…
We study Gaussian approximations to the distribution of a diffusion. The approximations are easy to compute: they are defined by two simple ordinary differential equations for the mean and the covariance. Time correlations can also be…