Related papers: Monotonically Decreasing Sequence of Divergences
We show that if a numerical method is posed as a sequence of operators acting on data and depending on a parameter, typically a measure of the size of discretization, then consistency, convergence and stability can be related by a…
In this paper, we study the statistical and geometrical properties of the Kullback-Leibler divergence with kernel covariance operators (KKL) introduced by Bach [2022]. Unlike the classical Kullback-Leibler (KL) divergence that involves…
Continuity, compactness, the spectrum and ergodic properties of the differentiation operator are investigated, when it acts in the Fr\'echet space of all Dirichlet series that are uniformly convergent in all half-planes $\{s \in \mathbb{C}…
Exploiting the geometric nature of statistical divergences, we devise a way to define associated induced uncertainty measures for discrete and finite probability distributions. We also report new uncertainty measures and discuss their…
We develop new adaptive algorithms for variational inequalities with monotone operators, which capture many problems of interest, notably convex optimization and convex-concave saddle point problems. Our algorithms automatically adapt to…
Kullback-Leibler (KL) divergence is a fundamental concept in information theory that quantifies the discrepancy between two probability distributions. In the context of Variational Autoencoders (VAEs), it serves as a central regularization…
This short note is on a property of the Kullback-Leibler (KL) divergence which indicates that independent Gaussian distributions minimize the KL divergence from given independent Gaussian distributions. The primary purpose of this note is…
The problem is sequence prediction in the following setting. A sequence x1,..., xn,... of discrete-valued observations is generated according to some unknown probabilistic law (measure) mu. After observing each outcome, it is required to…
Disagreement between two classifiers regarding the class membership of an observation in pattern recognition can be indicative of an anomaly and its nuance. As in general classifiers base their decision on class aposteriori probabilities,…
This article discusses the convergence properties of the Max Product and Max Min variants of Durrmeyer type exponential sampling series. We first establish pointwise and uniform convergence of both operators in the space of log uniformly…
We introduce the proximal optimal transport divergence, a novel discrepancy measure that interpolates between information divergences and optimal transport distances via an infimal convolution formulation. This divergence provides a…
When and why representations learned by different deep neural networks are similar is an active research topic. We choose to address these questions from the perspective of identifiability theory, which suggests that a measure of…
For a function defined on a convex set in a Euclidean space, midpoint convexity is the property requiring that the value of the function at the midpoint of any line segment is not greater than the average of its values at the endpoints of…
Statistical divergences (SDs), which quantify the dissimilarity between probability distributions, are a basic constituent of statistical inference and machine learning. A modern method for estimating those divergences relies on…
We consider divergence form elliptic operators in dimension $n\geq 2$ with $L^\infty$ coefficients. Although solutions of these operators are only H\"{o}lder continuous, we show that they are differentiable ($C^{1,\alpha}$) with respect to…
Bregman divergences are a class of distance-like comparison functions which play fundamental roles in optimization, statistics, and information theory. One important property of Bregman divergences is that they cause two useful formulations…
Group-invariant probability distributions appear in many data-generative models in machine learning, such as graphs, point clouds, and images. In practice, one often needs to estimate divergences between such distributions. In this work, we…
The most common way to sample from a probability distribution is to use Monte-Carlo methods. For distributions on a continuous state space, one can find diffusions with the target distribution as equilibrium measure, so that the state of…
A refinement of the multinomial distribution is presented where the number of inversions in the sequence of outcomes is tallied. This refinement of the multinomial distribution is its joint distribution with the number of inversions in the…
Bregman divergences play a central role in the design and analysis of a range of machine learning algorithms. This paper explores the use of Bregman divergences to establish reductions between such algorithms and their analyses. We present…