Related papers: Entropy and the Discrete Central Limit Theorem
We consider the central limit theorem for stable laws in the case of the standardized sum of independent and identically distributed random variables with regular probability density function. By showing decay of different entropy…
The Central Limit Theorem states that, in the limit of a large number of terms, an appropriately scaled sum of independent random variables yields another random variable whose probability distribution tends to a stable distribution. The…
The Central Limit Theorem (CLT) is one of the most fundamental results in statistics. It states that the standardized sample mean of a sequence of $n$ mutually independent and identically distributed random variables with finite first and…
For normalized sums $Z_n$ of i.i.d. random variables, we explore necessary and sufficient conditions which guarantee the normal approximation with respect to the R\'enyi divergence of infinite order. In terms of densities $p_n$ of $Z_n$,…
An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic…
New families of Fisher information and entropy power inequalities for sums of independent random variables are presented. These inequalities relate the information in the sum of $n$ independent random variables to the information contained…
We present some new nonparametric estimators of entropies and we establish almost sure consistency and central limit Theorems for some of the most important entropies in the discrete case. Our theorical results are validated by simulations.
This paper derives new bounds on the difference of the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal…
In this paper, we investigate the asymptotic stability of finite-dimensional stochastic integrable Hamiltonian systems via information entropy. Specifically, we establish the asymptotic vanishing of Shannon entropy difference (with…
We prove a central limit theorem for a sequence of random variables whose means are ambiguous and vary in an unstructured way. Their joint distribution is described by a set of measures. The limit is (not the normal distribution and is)…
In this article we review recent generalisations of the central limit theorem for the sum of specially correlated (or q-independent) variables, focusing on q greater or equal than 1. Specifically, this kind of correlation turns the…
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…
We adapt arguments concerning information-theoretic convergence in the Central Limit Theorem to the case of dependent random variables under Rosenblatt mixing conditions. The key is to work with random variables perturbed by the addition of…
We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for…
The central limit theorem provides the theoretical foundation for the universality of the normal distribution: under broad conditions, the asymptotic distribution of a sum of independent random variables approaches a Gaussian. Yet, physical…
We study the relative entropy between the empirical estimate of a discrete distribution and the true underlying distribution. If the minimum value of the probability mass function exceeds an $\alpha > 0$ (i.e. when the true underlying…
In order to characterize the fluctuation between the ergodic limit and the time-averaging estimator of a full discretization in a quantitative way, we establish a central limit theorem for the full discretization of the parabolic stochastic…
The discounted central limit theorem concerns the convergence of an infinite discounted sum of i.i.d. random variables to normality as the discount factor approaches $1$. We show that, using the Fourier metric on probability distributions,…
This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…
The standard central limit theorem with a Gaussian attractor for the sum of independent random variables may lose its validity in presence of strong correlations between the added random contributions. Here, we study this problem for…