Related papers: Stein transport for Bayesian inference
Variational inference is a technique that approximates a target distribution by optimizing within the parameter space of variational families. On the other hand, Wasserstein gradient flows describe optimization within the space of…
We analyze in a closed form the learning dynamics of stochastic gradient descent (SGD) for a single-layer neural network classifying a high-dimensional Gaussian mixture where each cluster is assigned one of two labels. This problem provides…
Worst-case generation plays a critical role in evaluating robustness and stress-testing systems under distribution shifts, in applications ranging from machine learning models to power grids and medical prediction systems. We develop a…
Variational Bayesian inference is an important machine-learning tool that finds application from statistics to robotics. The goal is to find an approximate probability density function (PDF) from a chosen family that is in some sense…
We build on the formalism developed in [arXiv:1906.08372v1] to propose new representations of solutions to Stein equations. We provide new uniform and non uniform bounds on these solutions (a.k.a.\ Stein factors). We use these…
Optimal transport (OT) provides powerful tools for comparing probability measures in various types. The Wasserstein distance which arises naturally from the idea of OT is widely used in many machine learning applications. Unfortunately,…
Using quasi-Newton methods in stochastic optimization is not a trivial task given the difficulty of extracting curvature information from the noisy gradients. Moreover, pre-conditioning noisy gradient observations tend to amplify the noise.…
Most stochastic optimization methods use gradients once before discarding them. While variance reduction methods have shown that reusing past gradients can be beneficial when there is a finite number of datapoints, they do not easily extend…
Effective uncertainty quantification is important for training modern predictive models with limited data, enhancing both accuracy and robustness. While Bayesian methods are effective for this purpose, they can be challenging to scale. When…
Regression analysis for responses taking values in general metric spaces has received increasing attention, particularly for settings with Euclidean predictors $X \in \mathbb{R}^p$ and non-Euclidean responses $Y$ in metric spaces. While…
We study the convergence of gradient flow for the training of deep neural networks. If Residual Neural Networks are a popular example of very deep architectures, their training constitutes a challenging optimization problem due notably to…
We propose a Stein variational gradient descent method to concurrently sparsify, train, and provide uncertainty quantification of a complexly parameterized model such as a neural network. It employs a graph reconciliation and condensation…
We develop a variant of Stein's method of comparison of generators to bound the Kolmogorov, total variation, and Wasserstein-1 distances between distributions on the real line. Our discrepancy is expressed in terms of the ratio of reverse…
We introduce an architecture for variational quantum algorithms that can be efficiently trained via parameter updates along exact geodesics on the Riemannian state manifold. This features a parameter-optimal circuit ansatz which supersedes…
We study the least-energy way to reshape a probability distribution when motion is constrained to a horizontal bundle, that is, optimal transport and distribution steering in sub-Riemannian geometry, motivated by density control over…
We present a continuation method that entails generating a sequence of transition probability density functions from the prior to the posterior in the context of Bayesian inference for parameter estimation problems. The characterization of…
In Bayesian applications, there is a huge interest in rapid and accurate estimation of the posterior distribution, particularly for high dimensional or hierarchical models. In this article, we propose to use optimization to solve for a…
We develop Bayesian predictive stacking for geostatistical models, where the primary inferential objective is to provide inference on the latent spatial random field and conduct spatial predictions at arbitrary locations. We exploit…
We study distribution-on-distribution regression problems in which a response distribution depends on multiple distributional predictors. Such settings arise naturally in applications where the outcome distribution is driven by several…
The sliced-Wasserstein flow is an evolution equation where a probability density evolves in time, advected by a velocity field computed as the average among directions in the unit sphere of the optimal transport displacements from its 1D…