Related papers: One-dimensional System Arising in Stochastic Gradi…
Moving boundary problems allow to model systems with phase transition at an inner boundary. Driven by problems in economics and finance, in particular modeling of limit order books, we consider a stochastic and non-linear extension of the…
A theoretical, and potentially also practical, problem with stochastic gradient descent is that trajectories may escape to infinity. In this note, we investigate uniform boundedness properties of iterates and function values along the…
In multistable dynamical systems driven by weak Gaussian noise, transitions between competing states are often assumed to pass via a saddle on the separating basin boundary. By contrast, we show that timescale separation can cause saddle…
In this note we consider SDEs of the type $\mathrm{d} X_t=[F (X_t) -A X_t] \mathrm{d} t +D \mathrm{d} W_t$ under the assumptions that $A$'s eigenvalues are all of positive real parts and $F (\cdot)$ has slower-than-linear growth rate. It is…
Point-gap topology, characterized by spectral winding numbers, is crucial to non-Hermitian topological phases and dramatically alters real-time dynamics. In this paper, we study the evolution of quantum particles in dissipative systems with…
We derive high-dimensional scaling limits and fluctuations for the online least-squares Stochastic Gradient Descent (SGD) algorithm by taking the properties of the data generating model explicitly into consideration. Our approach treats the…
Most results on Stochastic Gradient Descent (SGD) in the convex and smooth setting are presented under the form of bounds on the ergodic function value gap. It is an open question whether bounds can be derived directly on the last iterate…
It was previously shown by Davis and Drusvyatskiy that every Clarke critical point of a generic, semialgebraic (and more generally definable in an o-minimal structure), weakly convex function is lying on an active manifold and is either a…
We consider deterministic fast-slow dynamical systems on $\mathbb{R}^m\times Y$ of the form \[ \begin{cases} x_{k+1}^{(n)} = x_k^{(n)} + n^{-1} a(x_k^{(n)}) + n^{-1/\alpha} b(x_k^{(n)}) v(y_k)\;,\quad y_{k+1} = f(y_k)\;, \end{cases} \]…
Stochastic gradient algorithm is a key ingredient of many machine learning methods, particularly appropriate for large-scale learning.However, a major caveat of large data is their incompleteness.We propose an averaged stochastic gradient…
Recent years have seen increased interest in performance guarantees of gradient descent algorithms for non-convex optimization. A number of works have uncovered that gradient noise plays a critical role in the ability of gradient descent…
We study the unconstrained and the minimax saddle point variants of the convex multi-stage stochastic programming problem, where consecutive decisions are coupled through the objective functions, rather than through the constraints. We…
Stochastic gradient descent (SGD) is widely used in machine learning. Although being commonly viewed as a fast but not accurate version of gradient descent (GD), it always finds better solutions than GD for modern neural networks. In order…
We establish that first-order methods avoid saddle points for almost all initializations. Our results apply to a wide variety of first-order methods, including gradient descent, block coordinate descent, mirror descent and variants thereof.…
Exclusion processes in one dimension first appeared in the 70s and have since dragged much attention from communities in different domains: stochastic processes, out-of-equilibriums statistical physics, and more recently integrable systems.…
Starting from the overdamped Langevin dynamics in $\mathbb{R}^n$, $$ dX_t = -\nabla V(X_t) dt + \sqrt{2 \beta^{-1}} dW_t, $$ we consider a scalar Markov process $\xi_t$ which approximates the dynamics of the first component $X^1_t$. In the…
We consider the generalization error associated with stochastic gradient descent on a smooth convex function over a compact set. We show the first bound on the generalization error that vanishes when the number of iterations $T$ and the…
Let $f:\mathbb{D}\to\mathbb{C}$ be a bounded analytic function. A set $K\subset\mathbb{D}$ which contains the point $1$ in its boundary is called a convergence set for $f$ at $1$ if $f(z)$ converges to some value $\zeta$ as $z\to1$ with…
In this paper, we propose a variance-reduced primal-dual algorithm with Bregman distance for solving convex-concave saddle-point problems with finite-sum structure and nonbilinear coupling function. This type of problems typically arises in…
A game theory inspired methodology is proposed for finding a function's saddle points. While explicit descent methods are known to have severe convergence issues, implicit methods are natural in an adversarial setting, as they take the…