Related papers: Characterizations and Kullback-Leibler Divergence …
Extreme Value Theory plays an important role to provide approximation results for the extremes of a sequence of independent random variables when their distribution is unknown. An important one is given by the {generalised Pareto…
We consider the fundamental problem of estimating a discrete distribution on a domain of size $K$ with high probability in Kullback-Leibler divergence. We provide upper and lower bounds on the minimax estimation rate, which show that the…
The statistical distribution of the largest value drawn from a sample of a given size has only three possible shapes: it is either a Weibull, a Fr\'echet or a Gumbel extreme value distributions. I describe in this short review how to relate…
The asymptotic correspondence between the probability mass function of the $q$-deformed multinomial distribution and the $q$-generalised Kullback-Leibler divergence, also known as Tsallis relative entropy, is established. The probability…
Wide conditions are provided to guarantee asymptotic unbiasedness and L^2-consistency of the introduced estimates of the Kullback-Leibler divergence for probability measures in R^d having densities w.r.t. the Lebesgue measure. These…
The aim of this paper is to study certain properties of the Kullback-Leibler distance between two positive integer numbers or between two ideals. We present some results related the entropy of a positive integer number and the divergence of…
In algebraic statistics, the maximum likelihood degree of a statistical model refers to the number of solutions (counted with multiplicity) of the score equations over the complex field. In this paper, the maximum likelihood degree of the…
Estimating the Kullback-Leibler (KL) divergence between two distributions given samples from them is well-studied in machine learning and information theory. Motivated by considerations of multi-group fairness, we seek KL divergence…
Many thermodynamic relations involve inequalities, with equality if a process does not involve dissipation. In this article we provide equalities in which the dissipative contribution is shown to involve the relative entropy (a.k.a.…
This document shows how to obtain the Jacobian and Hessian matrices of the Kullback-Leibler divergence between two multivariate Gaussian distributions, using the first and second-order differentials. The presented derivations are based on…
The families of $f$-divergences (e.g. the Kullback-Leibler divergence) and Integral Probability Metrics (e.g. total variation distance or maximum mean discrepancies) are widely used to quantify the similarity between probability…
Learning word representations has garnered greater attention in the recent past due to its diverse text applications. Word embeddings encapsulate the syntactic and semantic regularities of sentences. Modelling word embedding as multi-sense…
We study extremal statistics and return intervals in stationary long-range correlated sequences for which the underlying probability density function is bounded and uniform. The extremal statistics we consider e.g., maximum relative to…
Given a random sample from a multivariate normal distribution whose covariance matrix is a Toeplitz matrix, we study the largest off-diagonal entry of the sample correlation matrix. Assuming the multivariate normal distribution has the…
The proposed paper discusses the problem of discrimination between close hypotheses about distributions belonging to the Gumbel maximum domain of attraction. The distinctive feature of the proposed work is using only k higher order…
We study the Kullback--Leibler (KL) divergence approximation theory of Gaussian mixture models (GMMs) by isolating an abstract mechanism behind several necessary-and-sufficient statements. The necessity direction is universal: if a density…
We investigate extreme values of Mahonian and Eulerian distributions arising from counting inversions and descents of random elements of finite Coxeter groups. To this end, we construct a triangular array of either distribution from a…
We study the distribution of spatial extrema of vorticity and the determinant of the strain rate tensor for a two-dimensional turbulent flow forced by a Kolmogorov forcing. The distribution of these quantities follow non-Gaussian behaviour…
We explain how the statistics of global observables in correlated systems can be related to extreme value problems and to Gumbel statistics. This relationship then naturally leads to the emergence of the generalized Gumbel distribution…
We consider asymptotic distributions of maximum deviations of sample covariance matrices, a fundamental problem in high-dimensional inference of covariances. Under mild dependence conditions on the entries of the data matrices, we establish…