Related papers: Generalised Pinsker Inequalities
Black box variational inference (BBVI) with reparameterization gradients triggered the exploration of divergence measures other than the Kullback-Leibler (KL) divergence, such as alpha divergences. In this paper, we view BBVI with…
Modern applications of Bayesian inference involve models that are sufficiently complex that the corresponding posterior distributions are intractable and must be approximated. The most common approximation is based on Markov chain Monte…
This paper introduces a variational approximation framework using direct optimization of what is known as the {\it scale invariant Alpha-Beta divergence} (sAB divergence). This new objective encompasses most variational objectives that use…
Selecting an appropriate divergence measure is a critical aspect of machine learning, as it directly impacts model performance. Among the most widely used, we find the Kullback-Leibler (KL) divergence, originally introduced in kinetic…
Variational inference with {\alpha}-divergences has been widely used in modern probabilistic machine learning. Compared to Kullback-Leibler (KL) divergence, a major advantage of using {\alpha}-divergences (with positive {\alpha} values) is…
We prove necessary optimality conditions of Euler-Lagrange type for generalized problems of the calculus of variations on time scales with a Lagrangian depending not only on the independent variable, an unknown function and its delta…
The problems in variation here concerned are such as to admit a continuous group (in Lie's sense); the conclusions that emerge from the corresponding differential equations find their most general expression in the theorems formulated in…
This paper introduces the $f$-divergence variational inference ($f$-VI) that generalizes variational inference to all $f$-divergences. Initiated from minimizing a crafty surrogate $f$-divergence that shares the statistical consistency with…
We deal with direct and inverse problems of the calculus of variations on arbitrary time scales. Firstly, using the Euler-Lagrange equation and the strengthened Legendre condition, we give a general form for a variational functional to…
There exist two different versions of the Kullback-Leibler divergence (K-Ld) in Tsallis statistics, namely the usual generalized K-Ld and the generalized Bregman K-Ld. Problems have been encountered in trying to reconcile them. A condition…
We study conditions on $f$ under which an $f$-divergence $D_f$ will satisfy $D_f \geq c_f V^2$ or $D_f \geq c_{2,f} V^2 + c_{4,f} V^4$, where $V$ denotes variational distance and the coefficients $c_f$, $c_{2,f}$ and $c_{4,f}$ are {\em best…
This paper examines the classical matching distribution arising in the "problem of coincidences". We generalise the classical matching distribution with a preliminary round of allocation where items are correctly matched with some fixed…
In this work we look at the original fractional calculus of variations problem in a somewhat different way. As a simple consequence, we show that a fractional generalization of a classical problem has a solution without any restrictions on…
In this paper, we study the statistical and geometrical properties of the Kullback-Leibler divergence with kernel covariance operators (KKL) introduced by Bach [2022]. Unlike the classical Kullback-Leibler (KL) divergence that involves…
A generalized constitutive relation error is proposed in an analogous form to Fenchel-Young inequality on the basis of the key idea of Legendre-Fenchel duality theory. The generalized constitutive relation error is linked with the global…
We use the definition of a fractional integral, recently proposed by Katugampola, to establish a generalization of the reverse Minkowski's inequality. We show two new theorems associated with this inequality, as well as state and show other…
In this paper, we introduce a split general quasi-variational inequality problem which is a natural extension of split variational inequality problem, quasi-variational and variational inequality problems in Hilbert spaces. Using projection…
We extend several recent results providing symmetry-based guarantees for variational inference (VI) with location-scale families. VI approximates a target density $p$ by the best match $q^*$ in a family $Q$ of tractable distributions that…
Estimating the Kullback-Leibler (KL) divergence between two distributions given samples from them is well-studied in machine learning and information theory. Motivated by considerations of multi-group fairness, we seek KL divergence…
Diverse inverse problems in imaging can be cast as variational problems composed of a task-specific data fidelity term and a regularization term. In this paper, we propose a novel learnable general-purpose regularizer exploiting recent…