Related papers: Minimum entropy of a log-concave variable for fixe…

An elementary proof is provided of sharp bounds for the varentropy of random vectors with log-concave densities, as well as for deviations of the information content from its mean. These bounds significantly improve on the bounds obtained…

Log-concave distributions include some important distributions such as normal distribution, exponential distribution and so on. In this note, we show inequalities between two Lp-norms for log-concave distributions on the Euclidean space.…

The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been…

We show that the minimum Renyi entropy output of a quantum channel is locally additive for Renyi parameter alpha>1. While our work extends the results of [10] (in which local additivity was proven for alpha=1), it is based on several new…

We consider the problem of approximating the empirical Shannon entropy of a high-frequency data stream under the relaxed strict-turnstile model, when space limitations make exact computation infeasible. An equivalent measure of entropy is…

We show that an information-theoretic property of Shannon's entropy power, known as concavity of entropy power, can be fruitfully employed to prove inequalities in sharp form. In particular, the concavity of entropy power implies the…

Additivity of minimal entropy output is proven for the class of quantum channels $\Lambda_t (A):=t A^{T}+(1-t)\tau (A)$ in the parameter range $-2/(d^2-2)\le t \le 1/(d+1)$.

We consider the LP in standard form min {c T x\,: Ax = b; x $\ge$ 0} and inspired by $\epsilon$-regularization in Optimal Transport, we introduce its $\epsilon$-regularization ''min {c T x + $\epsilon$ f (x)\,: Ax = b; x $\ge$ 0}'' via the…

Invariance entropy is a measure for the smallest data rate in a noiseless digital channel above which a controller that only receives state information through this channel is able to render a given subset of the state space invariant. In…

We prove the quantum conditional Entropy Power Inequality for quantum additive noise channels. This inequality lower bounds the quantum conditional entropy of the output of an additive noise channel in terms of the quantum conditional…

We describe five types of results concerning information and concentration of discrete random variables, and relationships between them, motivated by their counterparts in the continuous case. The results we consider are information…

From the output produced by a memoryless deletion channel from a uniformly random input of known length $n$, one obtains a posterior distribution on the channel input. The difference between the Shannon entropy of this distribution and that…

The behavior of the Kozachenko - Leonenko estimates for the (differential) Shannon entropy is studied when the number of i.i.d. vector-valued observations tends to infinity. The asymptotic unbiasedness and L^2-consistency of the estimates…

Using a sharp version of the reverse Young inequality, and a R\'enyi entropy comparison result due to Fradelizi, Madiman, and Wang, the authors are able to derive R\'enyi entropy power inequalities for log-concave random vectors when…

We propose a unified framework for likelihood-based regression modeling when the response variable has finite support. Our work is motivated by the fact that, in practice, observed data are discrete and bounded. The proposed methods assume…

We prove an exponential decay concentration inequality to bound the tail probability of the difference between the log-likelihood of discrete random variables on a finite alphabet and the negative entropy. The concentration bound we derive…

An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic…

The statistical analysis of data stemming from dynamical systems, including, but not limited to, time series, routinely relies on the estimation of information theoretical quantities, most notably Shannon entropy. To this purpose, possibly…

We establish the first known upper bound on the exact and Wyner's common information of $n$ continuous random variables in terms of the dual total correlation between them (which is a generalization of mutual information). In particular, we…

We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…