Related papers: R\'enyi Divergence and Kullback-Leibler Divergence
The existence of a unique Augustin mean and its invariance under the Augustin operator are established for arbitrary input distributions with finite Augustin information for channels with countably generated output $\sigma$-algebras. The…
We address an information-theoretic approach to noise and disturbance in quantum measurements. Properties of corresponding probability distributions are characterized by means of both the R\'{e}nyi and Tsallis entropies. Related…
We study convexity properties of R\'{e}nyi entropy as function of $\alpha>0$ on finite alphabets. We also describe robustness of the R\'{e}nyi entropy on finite alphabets, and it turns out that the rate of respective convergence depends on…
Variational techniques have been recently developed to find tighter bounds on the von Neumann entropy in a completely device-independent (DI) setting. This, in turn, has led to significantly improved key rates of DI protocols, in both the…
In many applications in biology, engineering and economics, identifying similarities and differences between distributions of data from complex processes requires comparing finite categorical samples of discrete counts. Statistical…
Concepts of information theory are increasingly used to characterize collective phenomena in condensed matter systems, such as the use of entanglement entropies to identify emergent topological order in interacting quantum many-body…
Integral representations of quantum relative entropy, and of the directional second and higher order derivatives of von Neumann entropy, are established, and used to give simple proofs of fundamental, known data processing inequalities: the…
We define a general notion of entropy in elementary, algebraic terms. Based on that, weak forms of a scalar product and a distance measure are derived. We give basic properties of these quantities, generalize the Cauchy-Schwarz inequality,…
Kullback-Leibler (KL) divergence is one of the most important divergence measures between probability distributions. In this paper, we prove several properties of KL divergence between multivariate Gaussian distributions. First, for any two…
Recent research has demonstrated significant achievable performance gains by exploiting circularity/non-circularity or propeness/improperness of complex-valued signals. In this paper, we investigate the influence of these properties on…
We establish sharp exponential deviation estimates of the information content as well as a sharp bound on the varentropy for the class of convex measures on Euclidean spaces. This generalizes a similar development for log-concave measures…
The asymptotic correspondence between the probability mass function of the $q$-deformed multinomial distribution and the $q$-generalised Kullback-Leibler divergence, also known as Tsallis relative entropy, is established. The probability…
This article develops an analytical framework for studying information divergences and likelihood ratios associated with Poisson processes and point patterns on general measurable spaces. The main results include explicit analytical…
We explore a family of information measures that stems from R\'enyi's $\alpha$-Divergences with $\alpha<0$. In particular, we extend the definition of Sibson's $\alpha$-Mutual Information to negative values of $\alpha$ and show several…
Consider the problems of computing the Augustin information and a R\'{e}nyi information measure of statistical independence, previously explored by Lapidoth and Pfister (IEEE Information Theory Workshop, 2018) and Tomamichel and Hayashi…
The book is structured into four main chapters. Chapter 1 introduces the foundational concepts of divergence measures, including the well-known Kullback-Leibler divergence and its limitations. It then presents a detailed exploration of…
Uncovering causal interdependencies from observational data is one of the great challenges of nonlinear time series analysis. In this paper, we discuss this topic with the help of information-theoretic concept known as R\'enyi information…
Defining suitable quantum extensions of classical divergences often poses a challenge due to the non-commutative nature of quantum information. In this work, we propose a new approach via what we call the layer cake representation. The…
Distributions of abundances or frequencies play an important role in many fields of science, from biology to sociology, as does the R\'enyi entropy, which measures the diversity of a statistical ensemble. We derive a mathematical relation…
The Pinsker inequality lower bounds the Kullback--Leibler divergence $D_{\textrm{KL}}$ in terms of total variation and provides a canonical way to convert $D_{\textrm{KL}}$ control into $\lVert \cdot \rVert_1$-control. Motivated by…