Related papers: Concavity of entropy under thinning

It is well-known that the Shannon entropies of some parameterized probability distributions are concave functions with respect to the parameter. In this paper we consider a family of such distributions (including the binomial, Poisson, and…

This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…

The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…

Building on work of Kontsevich, we introduce a definition of the entropy of a finite probability distribution in which the "probabilities" are integers modulo a prime p. The entropy, too, is an integer mod p. Entropy mod p is shown to be…

We prove new entropy inequalities for log concave and s-concave functions that strengthen and generalize recently established reverse log Sobolev and Poincare inequalities for such functions. This leads naturally to the concept of…

We show that Shannon's entropy--power inequality admits a strengthened version in the case in which the densities are log-concave. In such a case, in fact, one can extend the Blachman--Stam argument to obtain a sharp inequality for the…

Entropic uncertainty relations $H(A)+H(B)\geqslant \gamma$ give a nonzero lower bound $\gamma$ to the sum of the Shannon entropies $H$ of the outcome probabilities of incompatible observables $A$ and $B$. They are better than the…

$H$-theorem states that the entropy production is nonnegative and, therefore, the entropy of a closed system should monotonically change in time. In information processing, the entropy production is positive for random transformation of…

It is well known that the entropy $H(X)$ of a finite random variable is always greater or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tights bounds on…

If $u : \Omega\subset \mathbb{R}^d \to {\rm X}$ is a harmonic map valued in a metric space ${\rm X}$ and ${\sf E} : {\rm X} \to \mathbb{R}$ is a convex function, in the sense that it generates an ${\rm EVI}_0$-gradient flow, we prove that…

The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been…

We introduce a simple geometrical construction similar to covariant holographic entanglement entropy but with the addition of a new term proportional to boundary region volume. This new procedure has properties strongly resembling classical…

We show that an information-theoretic property of Shannon's entropy power, known as concavity of entropy power, can be fruitfully employed to prove inequalities in sharp form. In particular, the concavity of entropy power implies the…

The main result (roughly) is that if (H_i) converges weakly to H and if also f(H_i) converges weakly to f(H), for a single strictly convex continuous function f, then (H_i) must converge strongly to H. One application is that if f(pr(H)) =…

We establish a discrete analog of the R\'enyi entropy comparison due to Bobkov and Madiman. For log-concave variables on the integers, the min entropy is within log e of the usual Shannon entropy. Additionally we investigate the entropic…

Consider tossing a collection of coins, each fair or biased towards heads, and take the distribution of the total number of heads that result. It is natural to conjecture that this distribution should be 'more random' when each coin is…

We investigate the role of convexity in R\'enyi entropy power inequalities. After proving that a general R\'enyi entropy power inequality in the style of Bobkov-Chistyakov (2015) fails when the R\'enyi parameter $r\in(0,1)$, we show that…

In this paper we study certain properties of R\'{e}nyi entropy functionals $H_\alpha(\mathcal{P})$ on the space of probability distributions over $\mathbb{Z}_+$. Primarily, continuity and convergence issues are addressed. Some properties…

We revisit and prove some convexity inequalities for trace functions conjectured in the earlier part I. The main functional considered is \Phi_{p,q}(A_1,A_2,...,A_m) = (trace((\sum_{j=1}^m A_j^p)^{q/p}))^{1/q} for m positive definite…

We say that a subset of C^n is hypoconvex if its complement is the union of complex hyperplanes. Let D be the closed unit disk in C, T the unit circle. We prove two conjectures of Helton and Marshall. (See ``Frequency domain design and…