Related papers: On gradient descent-ascent flows in metric spaces

Gradient descent (GD) methods for the training of artificial neural networks (ANNs) belong nowadays to the most heavily employed computational schemes in the digital world. Despite the compelling success of such methods, it remains an open…

Stochastic gradient descent ascent (SGDA) and its variants have been the workhorse for solving minimax problems. However, in contrast to the well-studied stochastic gradient descent (SGD) with differential privacy (DP) constraints, there is…

In this note we continue the analysis of metric measure space with variable ricci curvature bounds. First, we study $(\kappa,N)$-convex functions on metric spaces where $\kappa$ is a lower semi-continuous function, and gradient flow curves…

We consider minimizing a nonconvex, smooth function $f$ on a Riemannian manifold $\mathcal{M}$. We show that a perturbed version of Riemannian gradient descent algorithm converges to a second-order stationary point (and hence is able to…

We consider gradient flow/gradient descent and heavy ball/accelerated gradient descent optimization for convex objective functions. In the gradient flow case, we prove the following: 1. If $f$ does not have a minimizer, the convergence…

We study the Wasserstein natural gradient in parametric statistical models with continuous sample spaces. Our approach is to pull back the $L^2$-Wasserstein metric tensor in the probability density space to a parameter space, equipping the…

This paper presents a new variational data assimilation (VDA) approach for the formal treatment of bias in both model outputs and observations. This approach relies on the Wasserstein metric stemming from the theory of optimal mass…

We derive bounds on the path length $\zeta$ of gradient descent (GD) and gradient flow (GF) curves for various classes of smooth convex and nonconvex functions. Among other results, we prove that: (a) if the iterates are linearly convergent…

We construct a non-local Benamou-Brenier-type transport distance on the space of stationary point processes and analyse the induced geometry. We show that our metric is a specific variant of the transport distance recently constructed in…

This paper provides results on Wasserstein gradient flows between measures on the real line. Utilizing the isometric embedding of the Wasserstein space $\mathcal P_2(\mathbb R)$ into the Hilbert space $L_2((0,1))$, Wasserstein gradient…

This paper considers the problem of solving systems of quadratic equations, namely, recovering an object of interest $\mathbf{x}^{\natural}\in\mathbb{R}^{n}$ from $m$ quadratic equations/samples…

We present a simple approach to study the one-dimensional pressureless Euler system via adhesion dynamics in the Wasserstein space of probability measures with finite quadratic moments. Starting from a discrete system of a finite number of…

We develop a unified theoretical framework for data-free one-step sampling from unnormalized target distributions based on Wasserstein gradient flows. For a broad class of standard f-divergence objectives, we show that the induced velocity…

We study a non-local version of the Cahn-Hilliard dynamics for phase separation in a two-component incompressible and immiscible mixture with linear mobilities. In difference to the celebrated local model with nonlinear mobility, it is only…

Optimal experimental design (OED) aims to choose the observations in an experiment to be as informative as possible, according to certain statistical criteria. In the linear case (when the observations depend linearly on the unknown…

Epoch gradient descent method (a.k.a. Epoch-GD) proposed by Hazan and Kale (2011) was deemed a breakthrough for stochastic strongly convex minimization, which achieves the optimal convergence rate of $O(1/T)$ with $T$ iterative updates for…

This paper develops the so-called Weighted Energy-Dissipation (WED) variational approach for the analysis of gradient flows in metric spaces. This focuses on the minimization of the parameter-dependent global-in-time functional of…

Finding constrained saddle points on Riemannian manifolds is significant for analyzing energy landscapes arising in physics and chemistry. Existing works have been limited to special manifolds that admit global regular level-set…

In this paper, we address the optimization problem of minimizing $Q(df_x)$ over a Hadamard manifold ${\cal M}$, where $f$ is a convex function on ${\cal M}$, $df_x$ is the differential of $f$ at $x \in {\cal M}$, and $Q$ is a function on…

The note considers normalized gradient descent (NGD), a natural modification of classical gradient descent (GD) in optimization problems. A serious shortcoming of GD in non-convex problems is that GD may take arbitrarily long to escape from…