Related papers: Jensen divergence based on Fisher's information
This work studies the Geometric Jensen-Shannon divergence, based on the notion of geometric mean of probability measures, in the setting of Gaussian measures on an infinite-dimensional Hilbert space. On the set of all Gaussian measures…
We show a general relation between the spatially disjoint product of probability density functions and the sum of their Fisher information metric tensors. We then utilise this result to give a method for constructing the probability density…
The coefficient of determination, known as $R^2$, is commonly used as a goodness-of-fit criterion for fitting linear models. $R^2$ is somewhat controversial when fitting nonlinear models, although it may be generalised on a case-by-case…
In this paper we have considered a single inequality having 11 known divergence measures. This inequality include measures like: Jeffryes-Kullback-Leiber J-divergence, Jensen-Shannon divergence (Burbea-Rao, 1982), arithmetic-geometric mean…
In this brief note we compute the Fisher information of a family of generalized normal distributions. Fisher information is usually defined for regular distributions, i.e. continuously differentiable (log) density functions whose support…
In transfer learning, training and testing data sets are drawn from different data distributions. The transfer generalization gap is the difference between the population loss on the target data distribution and the training loss. The…
A large class of modern probabilistic learning systems assumes symmetric distributions, however, real-world data tend to obey skewed distributions and are thus not always adequately modelled through symmetric distributions. To address this…
We provide a new perspective on Stein's so-called density approach by introducing a new operator and characterizing class which are valid for a much wider family of probability distributions on the real line. We prove an elementary…
The Jeffreys divergence is a renown symmetrization of the oriented Kullback-Leibler divergence broadly used in information sciences. Since the Jeffreys divergence between Gaussian mixture models is not available in closed-form, various…
In this paper we have considered a difference of Jensen's inequality for convex functions and proved some of its properties. In particular, we have obtained results for Csisz\'{a}r \cite{csi1} $f-$divergence. A result is established that…
The Renyi, Shannon and Fisher spreading lengths of the classical or hypergeometric orthogonal polynomials, which are quantifiers of their distribution all over the orthogonality interval, are defined and investigated. These…
Information distances like the Hellinger distance and the Jensen-Shannon divergence have deep roots in information theory and machine learning. They are used extensively in data analysis especially when the objects being compared are high…
There are three classical divergence measures in the literature on information theory and statistics, namely, Jeffryes-Kullback-Leiber's J-divergence, Sibson-Burbea-Rao's Jensen-Shannon divegernce and Taneja's arithemtic-geometric mean…
In this work, we define a family of probability densities involving the generalized trigonometric functions defined by Dr\'abek and Man\'asevich [1], which we name Generalized Trigonometric Densities. We show their relationship with the…
The Jensen's inequality plays a crucial role in the analysis of time-delay and sampled-data systems. Its conservatism is studied through the use of the Gr\"{u}ss Inequality. It has been reported in the literature that fragmentation (or…
The Fisher-Snedecor $\mathcal{F}$ distribution has been recently proposed as a more accurate and mathematically tractable composite fading model than traditional established models in some practical cases. In this paper, we firstly derive…
We introduce Fisher consistency in the sense of unbiasedness as a desirable property for estimators of class prior probabilities. Lack of Fisher consistency could be used as a criterion to dismiss estimators that are unlikely to deliver…
This paper gives upper and lower bounds on the gap in Jensen's inequality, i.e., the difference between the expected value of a function of a random variable and the value of the function at the expected value of the random variable. The…
The analytic information theory of discrete distributions was initiated in 1998 by C. Knessl, P. Jacquet and S. Szpankowski who addressed the precise evaluation of the Renyi and Shannon entropies of the Poisson, Pascal (or negative…
In proofs of L_2-differentiability, Lebesgue densities of a central distribution are often assumed right from the beginning. Generalizing Theorem 4.2 of Huber[81], we show that in the class of smooth parametric group models these densities…