Related papers: Bounds on Non-Symmetric Divergence Measures in Ter…
Minimizing divergence measures under a constraint is an important problem. We derive a sufficient condition that binary divergence measures provide lower bounds for symmetric divergence measures under a given triangular discrimination or…
Quantifying the difference between probability distributions is crucial in machine learning. However, estimating statistical divergences from empirical samples is challenging due to unknown underlying distributions. This work proposes the…
The concept of varentropy has been recently introduced as a dispersion index of the reliability of measure of information. In this paper, we introduce new measures of variability for two measures of uncertainty, the Kerridge inaccuracy…
In this paper, we introduce new classes of divergences by extending the definitions of the Bregman divergence and the skew Jensen divergence. These new divergence classes (g-Bregman divergence and skew g-Jensen divergence) satisfy some…
Estimating the Shannon entropy of a discrete distribution from which we have only observed a small sample is challenging. Estimating other information-theoretic metrics, such as the Kullback-Leibler divergence between two sparsely sampled…
Complex, high-dimensional data is ubiquitous across many scientific disciplines, including machine learning, biology, and the social sciences. One of the primary methods of visualizing these datasets is with two-dimensional scatter plots…
R\'enyi divergence is related to R\'enyi entropy much like Kullback-Leibler divergence is related to Shannon's entropy, and comes up in many settings. It was introduced by R\'enyi as a measure of information that satisfies almost the same…
Any physical system can be viewed from the perspective that information is implicitly represented in its state. However, the quantification of this information when it comes to complex networks has remained largely elusive. In this work, we…
Tight bounds for several symmetric divergence measures are derived in terms of the total variation distance. It is shown that each of these bounds is attained by a pair of 2 or 3-element probability distributions. An application of these…
Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback-Leibler (KL) divergence-based variational…
In this paper we study the quantum generalisation of the skew divergence, which is a dissimilarity measure between distributions introduced by L. Lee in the context of natural language processing. We provide an in-depth study of the quantum…
The book is structured into four main chapters. Chapter 1 introduces the foundational concepts of divergence measures, including the well-known Kullback-Leibler divergence and its limitations. It then presents a detailed exploration of…
Mutual Information (MI) is a fundamental measure of statistical dependence widely used in representation learning. While direct optimization of MI via its definition as a Kullback-Leibler divergence (KLD) is often intractable, many recent…
The Jensen inequality is a widely used tool in a multitude of fields, such as for example information theory and machine learning. It can be also used to derive other standard inequalities such as the inequality of arithmetic and geometric…
In a recent paper, the generalization of the Jensen Shannon divergence (JSD) in the context of quantum theory has been studied (Phys. Rev. A 72, 052310 (2005)). This distance between quantum states has shown to verify several of the…
Jensen-Shannon divergence is a well known multi-purpose measure of dissimilarity between probability distributions. It has been proven that the square root of this quantity is a true metric in the sense that, in addition to the basic…
We explore a family of information measures that stems from R\'enyi's $\alpha$-Divergences with $\alpha<0$. In particular, we extend the definition of Sibson's $\alpha$-Mutual Information to negative values of $\alpha$ and show several…
We extend the trace-logarithmic $S$-divergence from matrices to tracial $C^*$-algebras and finite von Neumann algebras, and show that its square root defines a metric on the invertible positive cone. We also prove an integral representation…
In 1938, Gini studied a mean having two parameters. Later, many authors studied properties of this mean. In particular, it contains the famous means as harmonic, geometric, arithmetic, etc. Here we considered a sequence of inequalities…
I. Sason obtained the tight bounds for symmetric divergence measures are derived by applying the results established by G. L. Gilardoni. In this article, we are going to report two kinds of extensions for the above results, namely classical…