Related papers: Jensen divergence based on Fisher's information
Jensen's inequality, attributed to Johan Jensen -- a Danish mathematician and engineer noted for his contributions to the theory of functions -- is a ubiquitous result in convex analysis, providing a fundamental lower bound for the…
Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…
We generalize the Jensen-Shannon divergence by considering a variational definition with respect to a generic mean extending thereby the notion of Sibson's information radius. The variational definition applies to any arbitrary distance and…
The Jensen inequality is a widely used tool in a multitude of fields, such as for example information theory and machine learning. It can be also used to derive other standard inequalities such as the inequality of arithmetic and geometric…
Parametric adversarial divergences, which are a generalization of the losses used to train generative adversarial networks (GANs), have often been described as being approximations of their nonparametric counterparts, such as the…
The shortcomings of the traditional univariate distributions in the past greatly encouraged mathematical statisticians to develop new generalizations of distributions. The New Generalized Fisk distribution, a unique distribution presented…
This short note presents results about the symmetric Jensen-Shannon divergence between two discrete mixture distributions $p_1$ and $p_2$. Specifically, for $i=1,2$, $p_i$ is the mixture of a common distribution $q$ and a distribution…
We first introduce the class of strictly quasiconvex and strictly quasiconcave Jensen divergences which are oriented (asymmetric) distances, and study some of their properties. We then define the strictly quasiconvex Bregman divergences as…
Bayesian predictive densities when the observed data $x$ and the target variable $y$ to be predicted have different distributions are investigated by using the framework of information geometry. The performance of predictive densities is…
In this paper, a new three-parameter lifetime distribution is introduced and many of its standard properties are discussed. These include shape of the probability density function, hazard rate function and its shape, quantile function,…
Consider two forms of sampling from a population: (i) drawing $s$ samples of $n$ elements with replacement and (ii) drawing a single sample of $ns$ elements. In this paper, under the setting where the descending order population frequency…
In this work we undertake a thorough study of the non-asymptotic properties of the vanilla generative adversarial networks (GANs). We prove an oracle inequality for the Jensen-Shannon (JS) divergence between the underlying density…
Shannon information entropy is a natural measure of probability (de)localization and thus (un)predictability in various procedures of data analysis for model systems. We pay particular attention to links between the Shannon entropy and the…
The mean of an unknown variance-$\sigma^2$ distribution $f$ can be estimated from $n$ samples with variance $\frac{\sigma^2}{n}$ and nearly corresponding subgaussian rate. When $f$ is known up to translation, this can be improved…
A two parameter generalization of Boltzmann-Gibbs-Shannon entropy based on natural logarithm is introduced. The generalization of the Shannon-Kinchinn axioms corresponding to the two parameter entropy is proposed and verified. We present…
In previous work the authors defined the k-th order simplicial distance between probability distributions which arises naturally from a measure of dispersion based on the squared volume of random simplices of dimension k. This theory is…
There are three classical divergence measures exist in the literature on information theory and statistics. These are namely, Jeffryes-Kullback-Leiber J-divergence. Sibson-Burbea-Rao Jensen-Shannon divegernce and Taneja arithemtic-geometric…
Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback-Leibler (KL) divergence-based variational…
There are three classical divergence measures exist in the literature on information theory and statistics. These are namely, Jeffryes-Kullback-Leiber J-divergence. Sibson-Burbea-Rao Jensen-Shannon divegernce and Taneja Arithmetic-Geometric…
Classically, Jensen's Inequality asserts that if $X$ is a compact convex set, and $f:K\to \mathbb{R}$ is a convex function, then for any probability measure $\mu$ on $K$, that $f(\text{bar}(\mu))\le \int f\;d\mu$, where $\text{bar}(\mu)$ is…