Related papers: On gradient descent-ascent flows in metric spaces
We study gradient flow on the multi-index regression problem for high-dimensional Gaussian data. Multi-index functions consist of a composition of an unknown low-rank linear projection and an arbitrary unknown, low-dimensional link…
We consider a one dimensional transport model with nonlocal velocity given by the Hilbert transform and develop a global well-posedness theory of probability measure solutions. Both the viscous and non-viscous cases are analyzed. Both in…
The paper surveys recent progresses in understanding the dynamics and loss landscape of the gradient flow equations associated to deep linear neural networks, i.e., the gradient descent training dynamics (in the limit when the step size…
Our work is motivated by a desire to study the theoretical underpinning for the convergence of stochastic gradient type algorithms widely used for non-convex learning tasks such as training of neural networks. The key insight, already…
We present a framework enabling variational data assimilation for gradient flows in general metric spaces, based on the minimizing movement (or Jordan-Kinderlehrer-Otto) approximation scheme. After discussing stability properties in the…
Gradient Descent (GD) is a powerful workhorse of modern machine learning thanks to its scalability and efficiency in high-dimensional spaces. Its ability to find local minimisers is only guaranteed for losses with Lipschitz gradients, where…
For addressing optimisation tasks on finite dimensional quantum systems, we give a comprehensive account of the foundations of gradient flows on Riemannian manifolds including new developments: we extend former results from Lie groups such…
We study the minimization of non-convex functionals over the Wasserstein space. While recent work has showed that perturbed Wasserstein gradient methods can avoid saddle points for benign landscapes, existing approaches remain essentially…
This paper studies the optimization of the KL functional on the Wasserstein space of probability measures, and develops a sampling framework based on Wasserstein gradient descent (WGD). We identify two important subclasses of the…
Over the past fifteen years, the theory of Wasserstein gradient flows of convex (or, more generally, semiconvex) energies has led to advances in several areas of partial differential equations and analysis. In this work, we extend the…
In this paper, we propose GT-GDA, a distributed optimization method to solve saddle point problems of the form: $\min_{\mathbf{x}} \max_{\mathbf{y}} \{F(\mathbf{x},\mathbf{y}) :=G(\mathbf{x}) + \langle \mathbf{y}, \overline{P} \mathbf{x}…
This article is concerned with the existence and the long time behavior of weak solutions to certain coupled systems of fourth-order degenerate parabolic equations of gradient flow type. The underlying metric is a Wasserstein-like…
This thesis analyze the Wasserstein gradient flow of a functional defined as a double convolution of a non-smooth repulsive interaction potential. To be more precise, the potential under investigation has a -|x| behavior close to the…
In centralized settings, it is well known that stochastic gradient descent (SGD) avoids saddle points and converges to local minima in nonconvex problems. However, similar guarantees are lacking for distributed first-order algorithms. The…
In high-dimensional time series analysis, Gaussian approximation (GA) schemes under various distance measures or on various collections of subsets of the Euclidean space play a fundamental role in a wide range of statistical inference…
Several widely-used first-order saddle-point optimization methods yield an identical continuous-time ordinary differential equation (ODE) that is identical to that of the Gradient Descent Ascent (GDA) method when derived naively. However,…
We study the equation of one-dimensional quasistatic nonlinear viscoelasticity with Dirichlet boundary conditions, in the particular case that the underlying dissipation geometry (provided by the viscosity) is comparable to the Bhattacharya…
The aim of this paper is twofold. Based on the geometric Wasserstein tangent space, we first introduce Wasserstein steepest descent flows. These are locally absolutely continuous curves in the Wasserstein space whose tangent vectors point…
Stochastic convex optimization is a basic and well studied primitive in machine learning. It is well known that convex and Lipschitz functions can be minimized efficiently using Stochastic Gradient Descent (SGD). The Normalized Gradient…
Wasserstein gradient flows on probability measures have found a host of applications in various optimization problems. They typically arise as the continuum limit of exchangeable particle systems evolving by some mean-field interaction…