Related papers: Divergence Measures Estimation and Its Asymptotic …

We derive the Kullback-Leibler divergence for the normal-gamma distribution and show that it is identical to the Bayesian complexity penalty for the univariate general linear model with conjugate priors. Based on this finding, we provide…

Fully Probabilistic design (FPD) is a powerful framework offering an elegant and unifying account of stochastic control, learning and decision-making. Here we introduce a generalized FPD framework, which we term as Tsallis FPD. Tsallis FPD…

Robust inference based on the minimization of statistical divergences has proved to be a useful alternative to classical techniques based on maximum likelihood and related methods. Basu et al. (1998) introduced the density power divergence…

The maximum likelihood method is the best-known method for estimating the probabilities behind the data. However, the conventional method obtains the probability model closest to the empirical distribution, resulting in overfitting. Then…

We consider the problem of estimating the density of observations taking values in classical or nonclassical spaces such as manifolds and more general metric spaces. Our setting is quite general but also sufficiently rich in allowing the…

It is well-known that in some situations it is not easy to compute the likelihood function as the datasets might be large or the model is too complex. In that contexts composite likelihood, derived by multiplying the likelihoods of subjects…

We prove that the $f$-divergences between univariate Cauchy distributions are all symmetric, and can be expressed as strictly increasing scalar functions of the symmetric chi-squared divergence. We report the corresponding scalar functions…

Any physical system can be viewed from the perspective that information is implicitly represented in its state. However, the quantification of this information when it comes to complex networks has remained largely elusive. In this work, we…

In this Note we introduce a new methodology for Bayesian inference through the use of $\phi$-divergences and the duality technique. The asymptotic laws of the estimates are established.

In this paper, we develop asymptotic theory for the mixing detection methodology proposed by M. Magdziarz and A. Weron [Physical Review E, 84:051138 (2011)]. The assumptions cover a broad family of Gaussian stochastic processes including…

The increased uncertainty and complexity of nonlinear systems have motivated investigators to consider generalized approaches to defining an entropy function. New insights are achieved by defining the average uncertainty in the probability…

In this short note, we prove an asymptotic expansion for the ratio of the Dirichlet density to the multivariate normal density with the same mean and covariance matrix. The expansion is then used to derive an upper bound on the total…

We characterize Martin-L\"of randomness and Schnorr randomness in terms of the merging of opinions, along the lines of the Blackwell-Dubins Theorem. After setting up a general framework for defining notions of merging randomness, we focus…

The probability density quantile (pdQ) carries essential information regarding shape and tail behavior of a location-scale family. Convergence of repeated applications of the pdQ mapping to the uniform distribution is investigated and new…

Random integers, sampled uniformly from $[1,x]$, share similarities with random permutations, sampled uniformly from $S_n$. These similarities include the Erd\H{o}s--Kac theorem on the distribution of the number of prime factors of a random…

We prove an isoperimetric inequality for probability measures $\mu$ on $\mathbb{R}^n$ with density proportional to $\exp(-\phi(\lambda | x|))$, where $|x|$ is the euclidean norm on $\mathbb{R}^n$ and $\phi$ is a non-decreasing convex…

Divergences are quantities that measure discrepancy between two probability distributions and play an important role in various fields such as statistics and machine learning. Divergences are non-negative and are equal to zero if and only…

We show that the $f$-divergence between any two densities of potentially different location-scale families can be reduced to the calculation of the $f$-divergence between one standard density with another location-scale density. It follows…

We study minimization of a parametric family of relative entropies, termed relative $\alpha$-entropies (denoted $\mathscr{I}_{\alpha}(P,Q)$). These arise as redundancies under mismatched compression when cumulants of compressed lengths are…

We study concentration inequalities for the Kullback--Leibler (KL) divergence between the empirical distribution and the true distribution. Applying a recursion technique, we improve over the method of types bound uniformly in all regimes…