Related papers: A closed-form formula for the Kullback-Leibler div…
This short note is on a property of the Kullback-Leibler (KL) divergence which indicates that independent Gaussian distributions minimize the KL divergence from given independent Gaussian distributions. The primary purpose of this note is…
A new integral representation is derived using a definite integral given by Cauchy and used to evaluate a number of integrals containing the finite series of special functions.
Mixture distributions arise in many application areas, for example as marginal distributions or convolutions of distributions. We present a method of constructing an easily tractable discrete mixture distribution as an approximation to a…
In this work, a mode of convergence for measurable functions is introduced. A related notion of Cauchy sequence is given and it is proved that this notion of convergence is complete in the sense that Cauchy sequences converge. Moreover, the…
We examine the estimation of the Kullback-Leibler (KL) divergence and the use of the goodness-of-fit test for multivariate continuous distributions. Our starting point is the maximum entropy principle for Shannon entropy: among all…
The Kullback-Leibler divergence or relative entropy is an information-theoretic measure between statistical models that play an important role in measuring a distance between random variables. In the study of complex systems, random fields…
A generalized Kullback-Leibler relative entropy is introduced starting with the symmetric Jackson derivative of the generalized overlap between two probability distributions. The generalization retains much of the structure possessed by the…
The probability density quantile (pdQ) carries essential information regarding shape and tail behavior of a location-scale family. Convergence of repeated applications of the pdQ mapping to the uniform distribution is investigated and new…
By calculating the Kullback-Leibler divergence between two probability measures belonging to different exponential families, we end up with a formula that generalizes the ordinary Fenchel-Young divergence. Inspired by this formula, we…
We show that the $f$-divergence between any two densities of potentially different location-scale families can be reduced to the calculation of the $f$-divergence between one standard density with another location-scale density. It follows…
For the regime-switching diffusion process with and without advection term we propose an integro-differential equation describing the densities of states continuously distributed over a segment. We demonstrate that there exists a…
Kullback-Leibler (KL) divergence is one of the most important divergence measures between probability distributions. In this paper, we prove several properties of KL divergence between multivariate Gaussian distributions. First, for any two…
\cite{tsagris2025a} proposed the generalized circular projected Cauchy (GCPC) distribution, whose special case is the wrapped Cauchy distribution. In this paper we first derive the relationship with the wrapped Cauchy distribution, and then…
We present a derivation of the Kullback Leibler (KL)-Divergence (also known as Relative Entropy) for the von Mises Fisher (VMF) Distribution in $d$-dimensions.
This article deals with goodness-of-fit test for the Cauchy distribution. Some tests based on Kullback-Leibler information are proposed, and shown to be consistent. Monte Carlo evidence indicates that the tests have satisfactory…
We study the problem of closeness testing for continuous distributions and its implications for causal discovery. Specifically, we analyze the sample complexity of distinguishing whether two multidimensional continuous distributions are…
This paper shows that large nonparametric classes of conditional multivariate densities can be approximated in the Kullback--Leibler distance by different specifications of finite mixtures of normal regressions in which normal means and…
We compute the expected value of the Kullback-Leibler divergence to various fundamental statistical models with respect to canonical priors on the probability simplex. We obtain closed formulas for the expected model approximation errors,…
Inferring and comparing complex, multivariable probability density functions is fundamental to problems in several fields, including probabilistic learning, network theory, and data analysis. Classification and prediction are the two faces…
We give two new simple characterizations of the Cauchy distribution by using the M\"obius and Mellin transforms. They also yield characterizations of the circular Cauchy distribution and the mixture Cauchy model.