Related papers: Thinning and Information Projections
In this paper we establish lower bounds on information divergence from a distribution to certain important classes of distributions as Gaussian, exponential, Gamma, Poisson, geometric, and binomial. These lower bounds are tight and for…
The hypergeometric distributions have many important applications, but they have not had sufficient attention in information theory. Hypergeometric distributions can be approximated by binomial distributions or Poisson distributions. In…
Renyi's "thinning" operation on a discrete random variable is a natural discrete analog of the scaling operation for continuous random variables. The properties of thinning are investigated in an information-theoretic context, especially in…
An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic…
If you take a superposition of n IID copies of a point process and thin that by a factor of 1/n, then the resulting process tends to a Poisson point process as n tends to infinity. We give a simple proof of this result that highlights its…
The minimum rate needed to accurately approximate a product distribution based on an unnormalized informational divergence is shown to be a mutual information. This result subsumes results of Wyner on common information and Han-Verd\'{u} on…
The Fisher information matrix (FIM) plays an important role in the analysis of parameter inference and system design problems. In a number of cases, however, the statistical data distribution and its associated information matrix are either…
We prove lower bounds on the error of any estimator for the mean of a real probability distribution under the knowledge that the distribution belongs to a given set. We apply these lower bounds both to parametric and nonparametric…
The law of large numbers for the empirical density for the pairs of uniformly distributed integers with a given greatest common divisor is a classic result in number theory. In this paper, we study the large deviations of the empirical…
Bounds of the accuracy of the normal approximation to the distribution of a sum of independent random variables are improved under relaxed moment conditions, in particular, under the absence of moments of orders higher than the second.…
In [Schuhmacher, Electron. J. Probab. 10 (2005), 165--201] estimates of the Barbour-Brown distance d_2 between the distribution of a thinned point process and the distribution of a Poisson process were derived by combining discretization…
In this paper, under mild assumptions, we derive a law of large numbers, a central limit theorem with an error estimate, an almost sure invariance principle and a variant of Chernoff bound in finite-state hidden Markov models. These limit…
Estimation of the $\phi$-divergence between two unknown probability distributions using empirical data is a fundamental problem in information theory and statistical learning. We consider a multi-variate generalization of the data dependent…
In this note we discuss additional properties of mixed Poisson distributions. We discuss the convergence of mixed Poisson distributions to its mixing distribution for the scaling parameter tending to infinity. Moreover, we obtain a central…
The mutual information between two jointly distributed random variables $X$ and $Y$ is a functional of the joint distribution $P_{XY},$ which is sometimes difficult to handle or estimate. A coarser description of the statistical behavior of…
Two new information-theoretic methods are introduced for establishing Poisson approximation inequalities. First, using only elementary information-theoretic techniques it is shown that, when $S_n=\sum_{i=1}^nX_i$ is the sum of the (possibly…
This article develops an analytical framework for studying information divergences and likelihood ratios associated with Poisson processes and point patterns on general measurable spaces. The main results include explicit analytical…
We study the problem of efficient compression of a stochastic source of probability distributions. It can be viewed as a generalization of Shannon's source coding problem. It has relation to the theory of common randomness, as well as to…
Under the sublinear expectation $\mathbb{E}[\cdot]:=\sup_{\theta\in \Theta} E_\theta[\cdot]$ for a given set of linear expectations $\{E_\theta: \theta\in \Theta\}$, we establish a new law of large numbers and a new central limit theorem…
We give a new lower bound for the minimal dispersion of a point set in the unit cube and its inverse function in the high dimension regime. This is done by considering only a very small class of test boxes, which allows us to reduce…