Related papers: Entropy and expansion
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
This paper is concerned with certain invariant random processes (called factors of IID) on infinite trees. Given such a process, one can assign entropies to different finite subgraphs of the tree. There are linear inequalities between these…
We consider families of finite quantum graphs of increasing size and we are interested in how eigenfunctions are distributed over the graph. As a measure for the distribution of an eigenfunction on a graph we introduce the entropy, it has…
Various lower bounds are established for the entropy of sums, products and their combinations. First, we derive a prime-field analogue of a version of the entropy power inequality established by Tao over torsion-free groups. Next, we prove…
New families of Fisher information and entropy power inequalities for sums of independent random variables are presented. These inequalities relate the information in the sum of $n$ independent random variables to the information contained…
In this paper we have shall generalize Shearer's entropy inequality and its recent extensions by Madiman and Tetali, and shall apply projection inequalities to deduce extensions of some of the inequalities concerning sums of sets of…
The Cheeger constant of a graph, or equivalently its coboundary expansion, quantifies the expansion of the graph. This notion assumes an implicit choice of a coefficient group, namely, $\mathbb{F}_2$. In this paper, we study Cheeger-type…
We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for…
Two new information-theoretic methods are introduced for establishing Poisson approximation inequalities. First, using only elementary information-theoretic techniques it is shown that, when $S_n=\sum_{i=1}^nX_i$ is the sum of the (possibly…
A lower bound on the R\'enyi differential entropy of a sum of independent random vectors is demonstrated in terms of rearrangements. For the special case of Boltzmann-Shannon entropy, this lower bound is better than that given by the…
We establish a lower bound on the entropy of weighted sums of (possibly dependent) random variables $(X_1, X_2, \dots, X_n)$ possessing a symmetric joint distribution. Our lower bound is in terms of the joint entropy of $(X_1, X_2, \dots,…
Lower bounds for the R\'enyi entropies of sums of independent random variables taking values in cyclic groups of prime order under permutations are established. The main ingredients of our approach are extended rearrangement inequalities in…
We review subbadditivity properties of Shannon entropy, in particular, from the Shearer's inequality we derive the "infimum rule" for actions of amenable groups. We briefly discuss applicability of the "infimum formula" to actions of other…
We derive an asymptotic lower bound on the Shannon entropy $H$ of sums of $N$ arbitrary iid discrete random variables. The derived bound $H \geq \frac{r(X)}{2}\log(N) + {\it cst}$ is given in terms of the incommensurability rank $r(X)$ of…
This paper considers the entropy of the sum of (possibly dependent and non-identically distributed) Bernoulli random variables. Upper bounds on the error that follows from an approximation of this entropy by the entropy of a Poisson random…
Following a growing number of studies that, over the past 15 years, have established entropy inequalities via ideas and tools from additive combinatorics, in this work we obtain a number of new bounds for the differential entropy of sums,…
A natural link between the notions of majorization and strongly Sperner posets is elucidated. It is then used to obtain a variety of consequences, including new R\'enyi entropy inequalities for sums of independent, integer-valued random…
The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if…
The Chernoff bound is an important inequality relation in probability theory. The original version of the Chernoff bound is to give an exponential decreasing bound on the tail distribution of sums of independent random variables. Recent…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…