Related papers: R\'enyi Divergence and Kullback-Leibler Divergence
The matrix-based R\'enyi's entropy allows us to directly quantify information measures from given data, without explicit estimation of the underlying probability distribution. This intriguing property makes it widely applied in statistical…
In this paper, we investigate the partition inequality, joint convexity, and Pinsker's inequality, for a divergence that generalizes the Tsallis Relative Entropy and Kullback-Leibler divergence. The generalized divergence is defined in…
The Renyi entropies constitute a family of information measures that generalizes the well-known Shannon entropy, inheriting many of its properties. They appear in the form of unconditional and conditional entropies, relative entropies or…
Information measures can be constructed from R\'enyi divergences much like mutual information from Kullback-Leibler divergence. One such information measure is known as Sibson $\alpha$-mutual information and has received renewed attention…
We explore properties of the $\chi^2$ and more general R\'enyi (Tsallis) distances to the normal law. In particular we provide necessary and sufficient conditions for the convergence to the normal law in the central limit theorem using…
This paper considers an information bottleneck problem with the objective of obtaining a most informative representation of a hidden feature subject to a R\'enyi entropy complexity constraint. The optimal bottleneck trade-off between…
Minimum divergence estimators provide a natural choice of estimators in a statistical inference problem. Different properties of various families of these divergence measures such as Hellinger distance, power divergence, density power…
In this chapter we shall discuss the recent progresses of information theoretic tools in the context of free and confined harmonic oscillator. Confined quantum systems have provided appreciable interest in areas of physics, chemistry,…
Divergences often play important roles for study in information science so that it is indispensable to investigate their fundamental properties. There is also a mathematical significance of such results. In this paper, we introduce some…
Entropies must correspond to mean values for them to be measurable. The Shannon entropy corresponds to the weighted arithmetic mean, whereas the Renyi entropy corresponds to the exponential mean. These means refer to code lengths, which are…
We study repeated independent Blackwell experiments; standard examples include drawing multiple samples from a population, or performing a measurement in different locations. In the baseline setting of a binary state of nature, we compare…
We propose an extension of the sandwiched R\'enyi relative $\alpha$-entropy to normal positive functionals on arbitrary von Neumann algebras, for the values $\alpha>1$. For this, we use Kosaki's definition of noncommutative $L_p$-spaces…
We demonstrate that the Renyi-2 entropy provides a natural measure of information for any multimode Gaussian state of quantum harmonic systems, operationally linked to the phase-space Shannon sampling entropy of the Wigner distribution of…
Here, we investigate the uncertainty of dynamical observables in classical systems manipulated by repeated measurements and feedback control; the precision should be enhanced in the presence of an external controller but limited by the…
Renyi's "thinning" operation on a discrete random variable is a natural discrete analog of the scaling operation for continuous random variables. The properties of thinning are investigated in an information-theoretic context, especially in…
For normalized sums $Z_n$ of i.i.d. random variables, we explore necessary and sufficient conditions which guarantee the normal approximation with respect to the R\'enyi divergence of infinite order. In terms of densities $p_n$ of $Z_n$,…
Entanglement criteria for an $n$-partite quantum system with continuous variables are formulated in terms of R\'{e}nyi entropies. R\'{e}nyi entropies are widely used as a good information measure due to many nice properties. Derived…
There are three classical divergence measures exist in the literature on information theory and statistics. These are namely, Jeffryes-Kullback-Leiber J-divergence. Sibson-Burbea-Rao Jensen-Shannon divegernce and Taneja arithemtic-geometric…
The generalized Kullback-Leibler divergence (K-Ld) in Tsallis statistics [constrained by the additive duality of generalized statistics (dual generalized K-Ld)] is here reconciled with the theory of Bregman divergences for expectations…
Both the Kullback-Leibler and the Tsallis divergence have a strong limitation: if the value $0$ appears in probability distributions $\left( p_{1},\cdots ,p_{n}\right)$ and $\left( q_{1},\cdots ,q_{n}\right)$, it must appear in the same…