Related papers: The case for shifting the Renyi Entropy
A new sharp inequality featuring the differential R\'enyi entropy, the R\'enyi divergence and the R\'enyi cross-entropy of a pair of probability density functions is established. The equality is reached when one of the probability density…
A new upper bound on the relative entropy is derived as a function of the total variation distance for probability measures defined on a common finite alphabet. The bound improves a previously reported bound by Csisz\'ar and Talata. It is…
The concept of Entropy plays a key role in Information Theory, Statistics, and Machine Learning.This paper introduces a new entropy measure, called the t-entropy, which exploits the concavity of the inverse-tan function. We analytically…
Concepts of information theory are increasingly used to characterize collective phenomena in condensed matter systems, such as the use of entanglement entropies to identify emergent topological order in interacting quantum many-body…
In the estimation theory context, we generalize the notion of Shannon's entropy power to the R\'{e}nyi-entropy setting. This not only allows to find new estimation inequalities, such as the R\'{e}nyi-entropy based De Bruijn identity,…
We propose R\'enyi information generating function and discuss its properties. A connection between the R\'enyi information generating function and the diversity index is proposed for discrete type random variables. The relation between the…
In this paper we study certain properties of R\'{e}nyi entropy functionals $H_\alpha(\mathcal{P})$ on the space of probability distributions over $\mathbb{Z}_+$. Primarily, continuity and convergence issues are addressed. Some properties…
R\'enyi divergence is related to R\'enyi entropy much like information divergence (also called Kullback-Leibler divergence or relative entropy) is related to Shannon's entropy, and comes up in many settings. It was introduced by R\'enyi as…
A novel definition of the conditional smooth Renyi entropy, which is different from that of Renner and Wolf, is introduced. It is shown that our definition of the conditional smooth Renyi entropy is appropriate to give lower and upper…
Entropy Estimation is an important problem with many applications in cryptography, statistic,machine learning. Although the estimators optimal with respect to the sample complexity have beenrecently developed, there are still some…
Estimation of Shannon and R\'enyi entropies of unknown discrete distributions is a fundamental problem in statistical property testing and an active research topic in both theoretical computer science and information theory. Tight bounds on…
Entropy and relative or cross entropy measures are two very fundamental concepts in information theory and are also widely used for statistical inference across disciplines. The related optimization problems, in particular the maximization…
This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…
The Renyi entropy is a generalization of the usual concept of entropy which depends on a parameter q. In fact, Renyi entropy is closely related to free energy. Suppose we start with a system in thermal equilibrium and then suddenly divide…
Lower bounds for the R\'enyi entropies of sums of independent random variables taking values in cyclic groups of prime order under permutations are established. The main ingredients of our approach are extended rearrangement inequalities in…
We associate to the p-th R\'enyi entropy a definition of entropy power, which is the natural extension of Shannon's entropy power and exhibits a nice behaviour along solutions to the p-nonlinear heat equation in $R^n$. We show that the…
Phase-space versions of quantum mechanics -- from Wigner's original distribution to modern discrete-qudit constructions -- represent some states with negative quasi-probabilities. Conventional Shannon and R\'enyi entropies become…
Since their introduction in the early sixties, the R\'enyi entropies have been used in many contexts, ranging from information theory to astrophysics, turbulence phenomena and others. In this note, we enlighten the main connections between…
Configurational entropy, or complexity, plays a critical role in characterizing disordered systems such as glasses, yet its measurement often requires significant computational resources. Recently, R\'enyi entropy, a one-parameter…
The Renyi entropies constitute a family of information measures that generalizes the well-known Shannon entropy, inheriting many of its properties. They appear in the form of unconditional and conditional entropies, relative entropies or…