Related papers: Generalised Pinsker Inequalities
In this paper, we study the strong convergence of an algorithm to solve the variational inequality problem which extends(Thong et al, Numerical Algorithms. 78, 1045-1060 (2018)). We have reduced and refined some of their algorithm's…
The problem of estimating the Kullback-Leibler divergence $D(P\|Q)$ between two unknown distributions $P$ and $Q$ is studied, under the assumption that the alphabet size $k$ of the distributions can scale to infinity. The estimation is…
We consider a mixed variational problem in real Hilbert spaces, defined on on the unbounded interval of time and governed by a history-dependent operator. We state the unique solvability of the problem, which follows from a general…
We derive a new variational formula for the R\'enyi family of divergences, $R_\alpha(Q\|P)$, between probability measures $Q$ and $P$. Our result generalizes the classical Donsker-Varadhan variational formula for the Kullback-Leibler…
In a regression setup with deterministic design, we study the pure aggregation problem and introduce a natural extension from the Gaussian distribution to distributions in the exponential family. While this extension bears strong…
Comparing probability distributions is an indispensable and ubiquitous task in machine learning and statistics. The most common way to compare a pair of Borel probability measures is to compute a metric between them, and by far the most…
Change of measure inequalities translate divergences between probability measures into explicit bounds on event probabilities, and play an important role in deriving probabilistic guarantees in learning theory, information theory, and…
Learning interpretable and human-controllable representations that uncover factors of variation in data remains an ongoing key challenge in representation learning. We investigate learning group-disentangled representations for groups of…
We consider the problem of parameter estimation in a Bayesian setting and propose a general lower-bound that includes part of the family of $f$-Divergences. The results are then applied to specific settings of interest and compared to other…
PAC-Bayes generalisation bounds are derived via change-of-measure inequalities that transfer concentration properties from a reference measure to all posterior measures. The specific choice of change of measure determines the assumptions…
In this paper we consider a generalized Kirchhoff? equation in a bounded domain under the effect of a sublinear nonlinearity. Under suitable assumptions on the data of the problem we show that, with a simple change of variable, the equation…
Pinsker's classical inequality asserts that the total variation $TV(\mu, \nu)$ between two probability measures is bounded by $\sqrt{ 2H(\mu|\nu)}$ where $H$ denotes the relative entropy (or Kullback-Leibler divergence). Considering the…
This paper is focused on $f$-divergences, consisting of three main contributions. The first one introduces integral representations of a general $f$-divergence by means of the relative information spectrum. The second part provides a new…
Main results and techniques of the fractional calculus of variations are surveyed. We consider variational problems containing Caputo derivatives and study them using both indirect and direct methods. In particular, we provide necessary…
Variational representations of $f$-divergences are central to many machine learning algorithms, with Lipschitz constrained variants recently gaining attention. Inspired by this, we define the Moreau-Yosida approximation of $f$-divergences…
This paper proposes two linear projection methods for supervised dimension reduction using only the first and second-order statistics. The methods, each catering to a different parameter regime, are derived under the general Gaussian model…
We extend the generic Bell inequalities suggested by Son, Lee, and Kim [Phys. Rev. Lett. 96, 060406 (2006)] to incorporate multiple observables for tripartite systems and introduce a geometric methodology for calculating classical upper…
We prove rates of convergence and robustness to prior misspecification within a Generalised Variational Inference (GVI) framework with bounded divergences. This addresses a significant open challenge for GVI and Federated GVI that employ a…
Kullback-Leibler (KL) divergence is one of the most important divergence measures between probability distributions. In this paper, we prove several properties of KL divergence between multivariate Gaussian distributions. First, for any two…
We prove existence of positive solutions to a nonlinear fractional boundary value problem. Then, under some mild assumptions on the nonlinear term, we obtain a smart generalization of Lyapunov's inequality. The new results are illustrated…