Related papers: Sum decomposition of divergence into three diverge…
There are many information and divergence measures exist in the literature on information theory and statistics. The most famous among them are Kullback-Leibler (1951) relative information and Jeffreys (1951) J-divergence. Sibson (1969)…
We define two recursive functions obtained by decomposition of a given interval into four close parts and prove two lemmas which determine features of these functions.
The discrepancy function measures the deviation of the empirical distribution of a point set in $[0,1]^d$ from the uniform distribution. In this paper, we study the classical discrepancy function with respect to the BMO and exponential…
Quantifying how distinguishable two stochastic processes are lies at the heart of many fields, such as machine learning and quantitative finance. While several measures have been proposed for this task, none have universal applicability and…
Certain notions of convergence of sequences functions such as pointwise convergence and (uniform) convergence on compact or bounded sets come from suitable topological function spaces; see [1]. Under certain conditions these topologies…
Divergence is not only an important mathematical concept in information theory, but also applied to machine learning problems such as low-dimensional embedding, manifold learning, clustering, classification, and anomaly detection. We…
In this paper we introduce the intuitive notion of trivergence of probability distributions (TPD). This notion allow us to calculate the similarity among triplets of objects. For this computation, we can use the well known measures of…
We revisit the mathematical foundations of proper scoring rules (PSRs) and Bregman divergences and present their characteristic properties in a unified theoretical framework. In many situations it is preferable not to generate a PSR…
The Jensen-Shannon divergence is a renown bounded symmetrization of the Kullback-Leibler divergence which does not require probability densities to have matching supports. In this paper, we introduce a vector-skew generalization of the…
In this paper, we introduce directed networks called `divergence network' in order to perform graphical calculation of divergence functions. By using the divergence networks, we can easily understand the geometric meaning of calculation…
Exponential families are statistical models which are the workhorses in statistics, information theory, and machine learning among others. An exponential family can either be normalized subtractively by its cumulant or free energy function…
In this paper we shall consider some famous means such as arithmetic, harmonic, geometric, root square mean, etc. Considering the difference of these means, we can establish. some inequalities among them. Interestingly, the difference of…
The idea of slicing divergences has been proven to be successful when comparing two probability measures in various machine learning applications including generative modeling, and consists in computing the expected value of a `base…
Recent success in contrastive learning has sparked growing interest in more effectively leveraging multiple augmented views of data. While prior methods incorporate multiple views at the loss or feature level, they primarily capture…
We introduce a novel family of distances, called the chord gap divergences, that generalizes the Jensen divergences (also called the Burbea-Rao distances), and study its properties. It follows a generalization of the celebrated statistical…
This paper extends the asymmetric Kullback-Leibler divergence and symmetric Jensen-Shannon divergence from two probability measures to the case of two sets of probability measures. We establish some fundamental properties of these…
In this article we give some improvements and generalizations of the famous Jensen's and Jensen-Mercer inequalities for twice differentiable functions, where convexity property of the target function is not assumed in advance. They…
In this paper, the concept of the classical $f$-divergence for a pair of measures is extended to the mixed $f$-divergence for multiple pairs of measures. The mixed $f$-divergence provides a way to measure the difference between multiple…
A formal sum $\sum_n f(S_n)$ may be seen as the integral $\int f dN$ with respect to random point process $N(A)=|\{n:S_n\in A\}|$. We study its convergence beyond the well known context of Lebesgue integrable functions, admitting…
In this paper, the concept of the classical $f$-divergence (for a pair of measures) is extended to the mixed $f$-divergence (for multiple pairs of measures). The mixed $f$-divergence provides a way to measure the difference between multiple…