Related papers: Divergence and Deformed Exponential Family
Recent advances in deep learning have brought to the fore models that can make multiple computational steps in the service of completing a task; these are capable of describ- ing long-term dependencies in sequential data. Novel recurrent…
In this dissertation, an abstract formalism extending information geometry is introduced. This framework encompasses a broad range of modelling problems, including possible applications in machine learning and in the information theoretical…
Structure and properties of families of critical points for classes of functions $W(z,\bar{z})$ obeying the elliptic Euler-Poisson-Darboux equation $E(1/2,1/2)$ are studied. General variational and differential equations governing the…
Discrete normal distributions are defined as the distributions with prescribed means and covariance matrices which maximize entropy on the integer lattice support. The set of discrete normal distributions form an exponential family with…
This document shows how to obtain the Jacobian and Hessian matrices of the Kullback-Leibler divergence between two multivariate Gaussian distributions, using the first and second-order differentials. The presented derivations are based on…
The probability density quantile (pdQ) carries essential information regarding shape and tail behavior of a location-scale family. Convergence of repeated applications of the pdQ mapping to the uniform distribution is investigated and new…
The aim of this paper is to study certain properties of the Kullback-Leibler distance between two positive integer numbers or between two ideals. We present some results related the entropy of a positive integer number and the divergence of…
Testing whether two multivariate samples exhibit the same extremal behavior is an important problem in various fields including environmental and climate sciences. While several ad-hoc approaches exist in the literature, they often lack…
We prove the large deviation principle (LDP) for posterior distributions arising from subfamilies of full exponential families, allowing misspecification of the model. Moreover, motivated by the so-called inverse Sanov Theorem (see e.g.…
In this work, we study generalized entropies and information geometry in a group-theoretical framework. We explore the conditions that ensure the existence of some natural properties and at the same time of a group-theoretical structure for…
Expectation maximization (EM) is the default algorithm for fitting probabilistic models with missing or latent variables, yet we lack a full understanding of its non-asymptotic convergence properties. Previous works show results along the…
Exponential families and mixture families are parametric probability models that can be geometrically studied as smooth statistical manifolds with respect to any statistical divergence like the Kullback-Leibler (KL) divergence or the…
The families of $f$-divergences (e.g. the Kullback-Leibler divergence) and Integral Probability Metrics (e.g. total variation distance or maximum mean discrepancies) are widely used to quantify the similarity between probability…
The Jensen-Shannon divergence is a renown bounded symmetrization of the unbounded Kullback-Leibler divergence which measures the total Kullback-Leibler divergence to the average mixture distribution. However the Jensen-Shannon divergence…
The $\alpha$-divergences include the well-known Kullback-Leibler divergence, Hellinger distance and $\chi^2$-divergence. In this paper, we derive differential and integral relations between the $\alpha$-divergences that are generalizations…
Wide conditions are provided to guarantee asymptotic unbiasedness and L^2-consistency of the introduced estimates of the Kullback-Leibler divergence for probability measures in R^d having densities w.r.t. the Lebesgue measure. These…
The Kullback-Leibler divergence or relative entropy is an information-theoretic measure between statistical models that play an important role in measuring a distance between random variables. In the study of complex systems, random fields…
Here we investigate meaningful families of vector bundles on a very general polarized $K3$ surface $(X,H)$ and on the corresponding Hyper--Kaehler variety given by the Hilbert scheme of points $X^{[k]}:= {\rm Hilb}^k(X)$, for any integer $k…
Exponential family plays an important role in information geometry. In arXiv:1811.01394, we introduced a method to construct an exponential family $\mathcal{P}=\{p_\theta\}_{\theta\in\Theta}$ on a homogeneous space $G/H$ from a pair…
We give an overview of statistical models and likelihood, together with two of its variants: penalized and hierarchical likelihood. The Kullback-Leibler divergence is referred to repeatedly, for defining the misspecification risk of a…