Related papers: Prove Costa's Entropy Power Inequality and High Or…
This paper addresses the correspondence between linear inequalities of Shannon entropy and differential entropy for sums of independent group-valued random variables. We show that any balanced (with the sum of coefficients being zero)…
We propose an extension of the quantum entropy power inequality for finite dimensional quantum systems, and prove a conditional quantum entropy power inequality by using the majorization relation as well as the concavity of entropic…
The entropic moments of the probability density of a quantum system in position and momentum spaces describe not only some fundamental and/or experimentally accessible quantities of the system, but also the entropic uncertainty measures of…
This paper is on developing some computer-assisted proof methods involving non-classical inequalities for Shannon entropy. Two areas of the applications of information inequalities are studied: Secret sharing schemes and hat guessing games.…
Expressions for (EPI Shannon type) Divergence-Power Inequalities (DPI) in two cases (time-discrete and band-limited time-continuous) of stationary random processes are given. The new expressions connect the divergence rate of the sum of…
Estimation of Shannon and R\'enyi entropies of unknown discrete distributions is a fundamental problem in statistical property testing and an active research topic in both theoretical computer science and information theory. Tight bounds on…
This paper studies the problem of estimating the differential entropy $h(S+Z)$, where $S$ and $Z$ are independent $d$-dimensional random variables with $Z\sim\mathcal{N}(0,\sigma^2 \mathrm{I}_d)$. The distribution of $S$ is unknown, but $n$…
The entropic region is formed by the collection of the Shannon entropies of all subvectors of finitely many jointly distributed discrete random variables. For four or more variables, the structure of the entropic region is mostly unknown.…
We produce a probabilistic space from logic, both classical and quantum, which is in addition partially ordered in such a way that entropy is monotone. In particular do we establish the following equation: Quantitative Probability = Logic +…
For the inertial Qian-Sheng model of nematic liquid crystals in the $Q$-tensor framework, we illustrate the roles played by the entropy inequality and energy dissipation in the well-posedness of smooth solutions when we employ energy…
We show that there is equality in Shannon's Entropy Power Inequality (EPI) if and only if the random variables involved are Gaussian, assuming nothing beyond the existence of differential entropies. This is done by justifying de Bruijn's…
Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must be of Boltzmann-Gibbs type, $S=-\sum_i p_i \log p_i$. Here we note that in physical systems one of these axioms may be violated. For non-ergodic…
Reliable data-driven estimation of Shannon entropy from small data sets, where the number of examples is potentially smaller than the number of possible outcomes, is a critical matter in several applications. In this paper, we introduce a…
Let $X$ be an arbitrary continuous random variable and $Z$ be an independent Gaussian random variable with zero mean and unit variance. For $t~>~0$, Costa proved that $e^{2h(X+\sqrt{t}Z)}$ is concave in $t$, where the proof hinged on the…
A lower bound on the R\'enyi differential entropy of a sum of independent random vectors is demonstrated in terms of rearrangements. For the special case of Boltzmann-Shannon entropy, this lower bound is better than that given by the…
The efficiency of Monte Carlo samplers is dictated not only by energetic effects, such as large barriers, but also by entropic effects that are due to the sheer volume that is sampled. The latter effects appear in the form of an entropic…
The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been…
Many partially-successful attempts have been made to find the most natural discrete-variable version of Shannon's entropy power inequality (EPI). We develop an axiomatic framework from which we deduce the natural form of a discrete-variable…
We introduce a notion of invariance entropy for uncertain control systems, which is, roughly speaking, the exponential growth rate of "branches" of "trees" that are formed by controls and are necessary to achieve invariance of controlled…
When two independent analog signals, X and Y are added together giving Z=X+Y, the entropy of Z, H(Z), is not a simple function of the entropies H(X) and H(Y), but rather depends on the details of X and Y's distributions. Nevertheless, the…