Related papers: Computing the Kullback-Leibler Divergence between …
This paper presents an analysis of the Theil and Atkinson index estimators for gamma populations, highlighting the presence of bias in both cases. Theoretical expressions for the biases are obtained, and bias-corrected estimators are…
This paper generalizes beta divergence beyond its classical form associated with power variance functions of Tweedie models. Generalized form is represented by a compact definite integral as a function of variance function of the…
The generalized Kullback-Leibler divergence (K-Ld) in Tsallis statistics [constrained by the additive duality of generalized statistics (dual generalized K-Ld)] is here reconciled with the theory of Bregman divergences for expectations…
To characterize the Kullback-Leibler divergence and Fisher information in general parametrized hidden Markov models, in this paper, we first show that the log likelihood and its derivatives can be represented as an additive functional of a…
We propose a method to fuse posterior distributions learned from heterogeneous datasets. Our algorithm relies on a mean field assumption for both the fused model and the individual dataset posteriors and proceeds using a simple…
There are many applications that benefit from computing the exact divergence between 2 discrete probability measures, including machine learning. Unfortunately, in the absence of any assumptions on the structure or independencies within…
In this paper, we discuss a property of the Kullback--Leibler divergence measured between two models of the family of the location-scale distributions. We show that, if model $M_1$ and model $M_2$ are represented by location-scale…
This paper extends the asymmetric Kullback-Leibler divergence and symmetric Jensen-Shannon divergence from two probability measures to the case of two sets of probability measures. We establish some fundamental properties of these…
We propose a closed-form spectral framework for relative log-density estimation in linearly parameterized probabilistic models, including unnormalized and conditional models. This is achieved by representing the Kullback-Leibler (KL)…
Gaussian Processes and the Kullback-Leibler divergence have been deeply studied in Statistics and Machine Learning. This paper marries these two concepts and introduce the local Kullback-Leibler divergence to learn about intervals where two…
We generalize the family of $\alpha$-divergences using a pair of strictly comparable weighted means. In particular, we obtain the $1$-divergence in the limit case $\alpha\rightarrow 1$ (a generalization of the Kullback-Leibler divergence)…
This work presents an infinite-dimensional generalization of the correspondence between the Kullback-Leibler and R\'enyi divergences between Gaussian measures on Euclidean space and the Alpha Log-Determinant divergences between symmetric,…
In this paper we propose an objective Bayesian estimation approach for the parameters of the generalized gamma distribution. Various reference priors are obtained, but showing that they lead to improper posterior distributions. We overcome…
How much one has learned from an experiment is quantifiable by the information gain, also known as the Kullback-Leibler divergence. The narrowing of the posterior parameter distribution $P(\theta|D)$ compared with the prior parameter…
We approximate a given rational spectral density by one that is consistent with prescribed second-order statistics. Such an approximation is obtained by minimizing a suitable distance from the given spectrum and under the constraints…
We present a derivation of the Kullback Leibler (KL)-Divergence (also known as Relative Entropy) for the von Mises Fisher (VMF) Distribution in $d$-dimensions.
The gamma difference distribution is defined as the difference of two gamma distributions, with in general different shape and rate parameters. Starting with knowledge of the corresponding characteristic function, a second order linear…
Inferring and comparing complex, multivariable probability density functions is fundamental to problems in several fields, including probabilistic learning, network theory, and data analysis. Classification and prediction are the two faces…
We obtain exact formulas for the cumulative distribution function of the variance-gamma distribution, as infinite series involving the modified Bessel function of the second kind and the modified Lommel function of the first kind. From…
The gamma distribution is a useful model for small area prediction of a skewed response variable. We study the use of the gamma distribution for small area prediction. We emphasize a model, called the gamma-gamma model, in which the area…