相关论文: Entropies, cross-entropies and R\'enyi divergence:…

We consider the maximum entropy problems associated with R\'enyi $Q$-entropy, subject to two kinds of constraints on expected values. The constraints considered are a constraint on the standard expectation, and a constraint on the…

We propose R\'enyi information generating function and discuss its properties. A connection between the R\'enyi information generating function and the diversity index is proposed for discrete type random variables. The relation between the…

Numerous entropy-type characteristics (functionals) generalizing R\'enyi entropy are widely used in mathematical statistics, physics, information theory, and signal processing for characterizing uncertainty in probability distributions and…

Many axiomatic definitions of entropy, such as the R\'enyi entropy, of a random variable are closely related to the $\ell_{\alpha}$-norm of its probability distribution. This study considers probability distributions on finite sets, and…

The de Bruijn identity states that Fisher information is equal to a half of the time-derivative of Shannon differential entropy along heat flow. In the same spirit, a generalized version of Fisher information, which we term the…

This paper is twofold. In the first part, we present a refinement of the R\'enyi Entropy Power Inequality (EPI) recently obtained in \cite{BM16}. The proof largely follows the approach in \cite{DCT91} of employing Young's convolution…

A framework for deriving R\'enyi entropy-power inequalities (EPIs) is presented that uses linearization and an inequality of Dembo, Cover, and Thomas. Simple arguments are given to recover the previously known R\'enyi EPIs and derive new…

Conventional information-theoretic quantities assume access to probability distributions. Estimating such distributions is not trivial. Here, we consider function-based formulations of cross entropy that sidesteps this a priori estimation…

R\'enyi divergence is related to R\'enyi entropy much like information divergence (also called Kullback-Leibler divergence or relative entropy) is related to Shannon's entropy, and comes up in many settings. It was introduced by R\'enyi as…

We investigate quantum R\'enyi entropic quantities, specifically those derived from 'sandwiched' divergence. This divergence is one of several proposed R\'enyi generalisations of the quantum relative entropy. We may define R\'enyi…

Since their introduction in the early sixties, the R\'enyi entropies have been used in many contexts, ranging from information theory to astrophysics, turbulence phenomena and others. In this note, we enlighten the main connections between…

The R\'{e}nyi cross-entropy measure between two distributions, a generalization of the Shannon cross-entropy, was recently used as a loss function for the improved design of deep learning generative adversarial networks. In this work, we…

This paper develops systematic approaches to obtain $f$-divergence inequalities, dealing with pairs of probability measures defined on arbitrary alphabets. Functional domination is one such approach, where special emphasis is placed on…

This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…

Using a sharp version of the reverse Young inequality, and a R\'enyi entropy comparison result due to Fradelizi, Madiman, and Wang, the authors are able to derive R\'enyi entropy power inequalities for log-concave random vectors when…

We consider a probability distribution depending on a real parameter $x$. As functions of $x$, the R\'enyi entropy and the Tsallis entropy can be expressed in terms of the associated index of coincidence $S(x)$. We establish recurrence…

This paper explores some applications of a two-moment inequality for the integral of the $r$-th power of a function, where $0 < r< 1$. The first contribution is an upper bound on the R\'{e}nyi entropy of a random vector in terms of the two…

Following a recent proof of Shannon's entropy power inequality (EPI), a comprehensive framework for deriving various EPIs for the R\'enyi entropy is presented that uses transport arguments from normal densities and a change of variable by…

Two typical fixed-length random number generation problems in information theory are considered for general sources. One is the source resolvability problem and the other is the intrinsic randomness problem. In each of these problems, the…

We introduce a variant of the R\'enyi entropy definition that aligns it with the well-known H\"older mean: in the new formulation, the r-th order R\'enyi Entropy is the logarithm of the inverse of the r-th order H\"older mean. This brings…