Related papers: Remarks on the R\'{e}nyi Entropy of a sum of IID r…
This paper considers the entropy of the sum of (possibly dependent and non-identically distributed) Bernoulli random variables. Upper bounds on the error that follows from an approximation of this entropy by the entropy of a Poisson random…
Variational inference (VI) is a popular approach in Bayesian inference, that looks for the best approximation of the posterior distribution within a parametric family, minimizing a loss that is typically the (reverse) Kullback-Leibler (KL)…
We show that for any $\alpha>0$ the R\'enyi entropy of order $\alpha$ is minimized, among all symmetric log-concave random variables with fixed variance, either for a uniform distribution or for a two sided exponential distribution. The…
We extend a lower bound of Munshi on sums over divisors of a number $n$ which are less than a fixed power of $n$ from the squarefree case to the general case. In the process we prove a lower bound on the entropy of a geometric distribution…
We establish the first known upper bound on the exact and Wyner's common information of $n$ continuous random variables in terms of the dual total correlation between them (which is a generalization of mutual information). In particular, we…
This note investigates invariance principles for sums of N(nt) iid radom variables, where n is an integer, t is a positive real number and N(u) is a stochastic process with nonnegative integer values. We show that the sequence of sums of…
The entropy power inequality (EPI) provides lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have…
Many partially-successful attempts have been made to find the most natural discrete-variable version of Shannon's entropy power inequality (EPI). We develop an axiomatic framework from which we deduce the natural form of a discrete-variable…
R\'enyi entropy is a one-parameter generalization of Shannon entropy, which has been used in various fields of physics. Despite its wide applicability, the physical interpretations of the R\'enyi entropy are not widely known. In this paper,…
We study the statistical behaviour of quantum entanglement in bipartite systems over fermionic Gaussian states as measured by von Neumann entropy. The formulas of average von Neumann entropy with and without particle number constrains have…
Problems of probabilistic inference and decision making under uncertainty commonly involve continuous random variables. Often these are discretized to a few points, to simplify assessments and computations. An alternative approximation is…
A well-known result across information theory, machine learning, and statistical physics shows that the maximum entropy distribution under a mean constraint has an exponential form called the Gibbs-Boltzmann distribution. This is used for…
We introduce an alternative closed form lower bound on the Gaussian process ($\mathcal{GP}$) likelihood based on the R\'enyi $\alpha$-divergence. This new lower bound can be viewed as a convex combination of the Nystr\"om approximation and…
Entropy is a fundamental concept in equilibrium statistical mechanics, yet its origin in the non-equilibrium dynamics of isolated quantum systems is not fully understood. A strong consensus is emerging around the idea that the stationary…
Quantum information measures such as the entropy and the mutual information find applications in physics, e.g., as correlation measures. Generalizing such measures based on the R\'enyi entropies is expected to enhance their scope in…
This article concerns the non-asymptotic analysis of the singular values (and Lyapunov exponents) of Gaussian matrix products in the regime where $N,$ the number of term in the product, is large and $n,$ the size of the matrices, may be…
We study convexity properties of R\'{e}nyi entropy as function of $\alpha>0$ on finite alphabets. We also describe robustness of the R\'{e}nyi entropy on finite alphabets, and it turns out that the rate of respective convergence depends on…
We consider the optimal quantization problem with R\'enyi-$\alpha$-entropy constraints for centered Gaussian measures on a separable Banach space. For $\alpha = \infty$ we can compute the optimal quantization error by a moment on a ball.…
R\'enyi divergence is related to R\'enyi entropy much like information divergence (also called Kullback-Leibler divergence or relative entropy) is related to Shannon's entropy, and comes up in many settings. It was introduced by R\'enyi as…
We consider a possible generalization of the random matrix theory, which involves the maximization of Tsallis' $q$-parametrized entropy. We discuss the dependence of the spacing distribution on $q$ using a non-extensive generalization of…