Related papers: Entropy and the Discrete Central Limit Theorem
We derive an asymptotic lower bound on the Shannon entropy $H$ of sums of $N$ arbitrary iid discrete random variables. The derived bound $H \geq \frac{r(X)}{2}\log(N) + {\it cst}$ is given in terms of the incommensurability rank $r(X)$ of…
Entropy rate is a real valued functional on the space of discrete random sources which lacks a closed formula even for subclasses of sources which have intuitive parameterizations. A good way to overcome this problem is to examine its…
A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the…
Various lower bounds are established for the entropy of sums, products and their combinations. First, we derive a prime-field analogue of a version of the entropy power inequality established by Tao over torsion-free groups. Next, we prove…
We consider the generalized differential entropy of normalized sums of independent and identically distributed (IID) continuous random variables. We prove that the R\'{e}nyi entropy and Tsallis entropy of order $\alpha\ (\alpha>0)$ of the…
We show how a central limit theorem for Poisson model random polygons implies a central limit theorem for uniform model random polygons. To prove this implication, it suffices to show that in the two models, the variables in question have…
Motivated by the entropy computations relevant to the evaluation of decrease in entropy in bit reset operations, the authors investigate the deficit in an entropic inequality involving two independent random variables, one continuous and…
We adapt arguments concerning entropy-theoretic convergence from the independent case to the case of FKG random variables. FKG systems are chosen since their dependence structure is controlled through covariance alone, though in the sequel…
We describe five types of results concerning information and concentration of discrete random variables, and relationships between them, motivated by their counterparts in the continuous case. The results we consider are information…
We consider $n\times n$ random matrices $M_{n}=\sum_{\alpha =1}^{m}{\tau _{\alpha }}\mathbf{y}_{\alpha }\otimes \mathbf{y}_{\alpha }$, where $\tau _{\alpha }\in \mathbb{R}$, $\{\mathbf{y}_{\alpha }\}_{\alpha =1}^{m}$ are i.i.d. isotropic…
We make use of an entropic property to establish a convergence theorem (Main Theorem), which reveals that the conditional entropy measures the asymptotic Gaussianity. As an application, we establish the {\it entropic conditional central…
In this paper, we derive a central limit theorem for collections of weakly correlated random variables indexed by discrete metric spaces, where the correlation decays in the distance of the indices. The correlation structure we study…
Through certain appropriate constructions, we establish periodic solutions in distribution for some stochastic differential equations with infinite-dimensional Levy noise. Additionally, we obtain the corresponding periodic measures and…
In this paper, we give the central limit theorem and almost sure central limit theorem for products of some partial sums of independent identically distributed random variables.
In this letter, we give a concise, closed-form expression for the differential entropy of the sum of two independent, non-identically-distributed exponential random variables. The derivation is straightforward, but such a concise entropy…
It is a manuscript for results about entropic central limit theorem for independent sum under finite Poincar\'e constant conditions.
The generalized entropic measure, which is optimized by a given arbitrary distribution under the constraints on normalization of the distribution and the finite ordinary expectation value of a physical random quantity, is considered and its…
For a general class of lattice-spin systems, we prove that an abstract Gaussian concentration bound implies positivity of lower relative entropy density. As a consequence we obtain uniqueness of translation-invariant Gibbs measures from the…
We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and…
A central limit theorem is established for a sum of random variables belonging to a sequence of random fields. The fields are assumed to have zero mean conditional on the past history and to satisfy certain conditional $\alpha$-mixing…