Related papers: R\'enyi Divergence and Kullback-Leibler Divergence
Recently, an interesting quantity called the quantum Renyi divergence (or "sandwiched" Renyi relative entropy) was defined for pairs of positive semi-definite operators $\rho$ and $\sigma$. It depends on a parameter $\alpha$ and acts as a…
Relative entropy is a measure of distinguishability for quantum states, and plays a central role in quantum information theory. The family of Renyi entropies generalizes to Renyi relative entropies that include as special cases most entropy…
Entropy and relative or cross entropy measures are two very fundamental concepts in information theory and are also widely used for statistical inference across disciplines. The related optimization problems, in particular the maximization…
In this paper we have considered a single inequality having 11 known divergence measures. This inequality include measures like: Jeffryes-Kullback-Leiber J-divergence, Jensen-Shannon divergence (Burbea-Rao, 1982), arithmetic-geometric mean…
An uncertainty relation for the R\'enyi entropies of conjugate quantum observables is used to obtain a strong Heisenberg limit of the form ${\rm RMSE} \geq f(\alpha)/(\langle N\rangle+\frac12)$, bounding the root mean square error of any…
Quantum information measures such as the entropy and the mutual information find applications in physics, e.g., as correlation measures. Generalizing such measures based on the R\'enyi entropies is expected to enhance their scope in…
A set of classical or quantum states is equivalent to another one if there exists a pair of classical or quantum channels mapping either set to the other one. For dichotomies (pairs of states), this is closely connected to (classical or…
Entanglement R\'enyi-$\alpha$ entropy is an entanglement measure. It generalizes the entanglement of formation, and they coincide when $\alpha$ tends to 1. We derive analytical lower and upper bounds for the entanglement R\'enyi-$\alpha$…
Convexity is a key concept in information theory, namely via the many implications of Jensen's inequality, such as the non-negativity of the Kullback-Leibler divergence (KLD). Jensen's inequality also underlies the concept of Jensen-Shannon…
This paper gives improved R\'{e}nyi entropy power inequalities (R-EPIs). Consider a sum $S_n = \sum_{k=1}^n X_k$ of $n$ independent continuous random vectors taking values on $\mathbb{R}^d$, and let $\alpha \in [1, \infty]$. An R-EPI…
Universal relations that characterize the fluctuations of nonequilibrium systems are of fundamental importance. The thermodynamic and kinetic uncertainty relations impose upper bounds on the precision of currents solely by total entropy…
We investigate the analogy between the renormalization group (RG) and deep neural networks, wherein subsequent layers of neurons are analogous to successive steps along the RG. In particular, we quantify the flow of information by…
We discuss a relation between the Kantorovich-Wasserstein (KW) metric and the Kullback-Leibler (KL) divergence. The former is defined using the optimal transport problem (OTP) in the Kantorovich formulation. The latter is used to define…
Around the mean dimensions and rate-distortion functions, using some tools from local entropy theory this paper establishes the following main results: $(1)$ We prove that for non-ergodic measures associated with almost sure processes, the…
We establish sharp rates for propagation of chaos in R\'enyi divergences for interacting diffusion systems at stationarity. Building upon the entropic hierarchy established in Lacker (2023), we show that under strong isoperimetry and weak…
In a certain sense we generalize the recently introduced and extensively studied notion called quantum R\'enyi divergence (in another name, sandwiched R\'enyi relative entropy) and describe the structures of corresponding symmetries. More…
This paper presents a novel theoretical study of the general problem of multiple source adaptation using the notion of Renyi divergence. Our results build on our previous work [12], but significantly broaden the scope of that work in…
This paper studies a specific class of statistical divergences for spectral densities of time series: the spectral $\alpha$-R\'{e}nyi divergences, which include the Itakura-Saito divergence as a limiting case. The aim of this paper is to…
Pinsker's widely used inequality upper-bounds the total variation distance $||P-Q||_1$ in terms of the Kullback-Leibler divergence $D(P||Q)$. Although in general a bound in the reverse direction is impossible, in many applications the…
We show how to sample in parallel from a distribution $\pi$ over $\mathbb R^d$ that satisfies a log-Sobolev inequality and has a smooth log-density, by parallelizing the Langevin (resp. underdamped Langevin) algorithms. We show that our…