Related papers: An Entropy Power Inequality for Discrete Random Va…
The entropy power inequality (EPI) provides lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have…
We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for…
It is known that the Entropy Power Inequality (EPI) always holds if the random variables have density. Not much work has been done to identify discrete distributions for which the inequality holds with the differential entropy replaced by…
It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tight…
An extension of the entropy power inequality to the form $N_r^\alpha(X+Y) \geq N_r^\alpha(X) + N_r^\alpha(Y)$ with arbitrary independent summands $X$ and $Y$ in $\mathbb{R}^n$ is obtained for the R\'enyi entropy and powers $\alpha \geq…
When two independent analog signals, X and Y are added together giving Z=X+Y, the entropy of Z, H(Z), is not a simple function of the entropies H(X) and H(Y), but rather depends on the details of X and Y's distributions. Nevertheless, the…
Many partially-successful attempts have been made to find the most natural discrete-variable version of Shannon's entropy power inequality (EPI). We develop an axiomatic framework from which we deduce the natural form of a discrete-variable…
The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been…
Yet another simple proof of the entropy power inequality is given, which avoids both the integration over a path of Gaussian perturbation and the use of Young's inequality with sharp constant or R\'enyi entropies. The proof is based on a…
We show that there is equality in Shannon's Entropy Power Inequality (EPI) if and only if the random variables involved are Gaussian, assuming nothing beyond the existence of differential entropies. This is done by justifying de Bruijn's…
The variance and the entropy power of a continuous random variable are bounded from below by the reciprocal of its Fisher information through the Cram\'{e}r-Rao bound and the Stam's inequality respectively. In this note, we introduce the…
We show that $h_\infty(X+Y)\leq h_\infty(Z+W)$, where $X, Y$ are independent log-concave random variables, and $Z, W$ are exponential random variables having the same respective $\infty$-R\'enyi entropies. Analogs for integer-valued…
This paper gives improved R\'{e}nyi entropy power inequalities (R-EPIs). Consider a sum $S_n = \sum_{k=1}^n X_k$ of $n$ independent continuous random vectors taking values on $\mathbb{R}^d$, and let $\alpha \in [1, \infty]$. An R-EPI…
We derive an asymptotic lower bound on the Shannon entropy $H$ of sums of $N$ arbitrary iid discrete random variables. The derived bound $H \geq \frac{r(X)}{2}\log(N) + {\it cst}$ is given in terms of the incommensurability rank $r(X)$ of…
How low can the joint entropy of $n$ $d$-wise independent (for $d\ge2$) discrete random variables be, subject to given constraints on the individual distributions (say, no value may be taken by a variable with probability greater than $p$,…
Shannon's Entropy Power Inequality can be viewed as characterizing the minimum differential entropy achievable by the sum of two independent random variables with fixed differential entropies. The entropy power inequality has played a key…
Following a growing number of studies that, over the past 15 years, have established entropy inequalities via ideas and tools from additive combinatorics, in this work we obtain a number of new bounds for the differential entropy of sums,…
Expressions for (EPI Shannon type) Divergence-Power Inequalities (DPI) in two cases (time-discrete and band-limited time-continuous) of stationary random processes are given. The new expressions connect the divergence rate of the sum of…
We utilize a discrete version of the notion of degree of freedom to prove a sharp min-entropy-variance inequality for integer valued log-concave random variables. More specifically, we show that the geometric distribution minimizes the…
It is well known that the entropy $H(X)$ of a finite random variable is always greater or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tights bounds on…