Related papers: Monotonically Decreasing Sequence of Divergences
Divergence functions play a key role as to measure the discrepancy between two points in the field of machine learning, statistics and signal processing. Well-known divergences are the Bregman divergences, the Jensen divergences and the…
We propose a method to measure real-valued time series irreversibility which combines two differ- ent tools: the horizontal visibility algorithm and the Kullback-Leibler divergence. This method maps a time series to a directed network…
$f$-divergences, which quantify discrepancy between probability distributions, are ubiquitous in information theory, machine learning, and statistics. While there are numerous methods for estimating $f$-divergences from data, a limit…
This paper studies the learning of linear operators between infinite-dimensional Hilbert spaces. The training data comprises pairs of random input vectors in a Hilbert space and their noisy images under an unknown self-adjoint linear…
This paper develops a new divergence that generalizes relative entropy and can be used to compare probability measures without a requirement of absolute continuity. We establish properties of the divergence, and in particular derive and…
Divergence functions are interesting discrepancy measures. Even though they are not true distances, we can use them to measure how separated two points are. Curiously enough, when they are applied to random variables, they lead to a notion…
Diversities are an extension of the concept of a metric space which assign a non-negative value to every finite set of points, rather than just pairs. A general theory of diversities has been developed which exhibits many deep analogies to…
Exponential families are statistical models which are the workhorses in statistics, information theory, and machine learning among others. An exponential family can either be normalized subtractively by its cumulant or free energy function…
There are three classical divergence measures exist in the literature on information theory and statistics. These are namely, Jeffryes-Kullback-Leiber J-divergence. Sibson-Burbea-Rao Jensen-Shannon divegernce and Taneja Arithmetic-Geometric…
Integration at a point is a new kind of integration derived from integration over an interval in infinitesimal and infinity domains which are spaces larger than the reals. Consider a continuous monotonic divergent function that is…
To ensure stability of learning, state-of-the-art generalized policy iteration algorithms augment the policy improvement step with a trust region constraint bounding the information loss. The size of the trust region is commonly determined…
Mirror Descent is a popular algorithm, that extends Gradients Descent (GD) beyond the Euclidean geometry. One of its benefits is to enable strong convergence guarantees through smooth-like analyses, even for objectives with exploding or…
This paper introduces a comprehensive framework for complex-valued probability measures and explores their novel applications in information theory and statistical analysis. We define a complex probability measure as a phase-modulated…
The problem of f-divergence estimation is important in the fields of machine learning, information theory, and statistics. While several nonparametric divergence estimators exist, relatively few have known convergence properties. In…
Variational inference with {\alpha}-divergences has been widely used in modern probabilistic machine learning. Compared to Kullback-Leibler (KL) divergence, a major advantage of using {\alpha}-divergences (with positive {\alpha} values) is…
In this paper we have considered a single inequality having 11 known divergence measures. This inequality include measures like: Jeffryes-Kullback-Leiber J-divergence, Jensen-Shannon divergence (Burbea-Rao, 1982), arithmetic-geometric mean…
The forward-backward splitting algorithm is a popular operator-splitting method for solving monotone inclusion of the sum of a maximal monotone operator and a cocoercive operator. In this paper, we present a new convergence analysis of a…
This paper investigates first-order variable metric backward forward dynamical systems associated with monotone inclusion and convex minimization problems in real Hilbert space. The operators are chosen so that the backward-forward…
This thesis develops a new divergence that generalizes relative entropy and can be used to compare probability measures without a requirement of absolute continuity. We establish properties of the divergence, and in particular derive and…
Minimum divergence estimators provide a natural choice of estimators in a statistical inference problem. Different properties of various families of these divergence measures such as Hellinger distance, power divergence, density power…