Related papers: One-dimensional System Arising in Stochastic Gradi…

Let $(X_t)_{t\ge 0}$ be the stochastic process solution to the overdamped Langevin dynamics $$dX_t=-\nabla f(X_t) \, dt +\sqrt h \, dB_t$$ and let $\Omega \subset \mathbb R^d $ be the basin of attraction of a local minimum of $f: \mathbb…

This paper presents an identity between the multivariate and univariate saddlepoint approximations applied to sample path probabilities for a certain class of stochastic processes. This class, which we term the recursively compounded…

This paper focuses on stochastic saddle point problems with decision-dependent distributions. These are problems whose objective is the expected value of a stochastic payoff function and whose data distribution drifts in response to…

In this paper, we analyze the convergence rate of the gradient temporal difference learning (GTD) family of algorithms. Previous analyses of this class of algorithms use ODE techniques to prove asymptotic convergence, and to the best of our…

In this paper we fully describe the trajectory of gradient flow over diagonal linear networks in the limit of vanishing initialisation. We show that the limiting flow successively jumps from a saddle of the training loss to another until…

In non-smooth stochastic optimization, we establish the non-convergence of the stochastic subgradient descent (SGD) to the critical points recently called active strict saddles by Davis and Drusvyatskiy. Such points lie on a manifold $M$…

We study the convergence properties and escape dynamics of Stochastic Gradient Descent (SGD) in one-dimensional landscapes, separately considering infinite- and finite-variance noise. Our main focus is to identify the time scales on which…

We classify transcendental entire functions that are compositions of a polynomial and the exponential for which all singular values escape on disjoint rays. We focus on the case where the escape is degenerate in the sense that points from…

We study a discrete non-autonomous system whose autonomous counterpart (with the frozen bifurcation parameter) admits a saddle-node bifurcation, and in which the bifurcation parameter slowly changes in time and is characterized by a sweep…

We analyze the behavior of randomized coordinate gradient descent for nonconvex optimization, proving that under standard assumptions, the iterates almost surely escape strict saddle points. By formulating the method as a nonlinear random…

In this paper, we propose a method for bounding the probability that a stochastic differential equation (SDE) system violates a safety specification over the infinite time horizon. SDEs are mathematical models of stochastic processes that…

We study the scaling limits of stochastic gradient descent (SGD) with constant step-size in the high-dimensional regime. We prove limit theorems for the trajectories of summary statistics (i.e., finite-dimensional functions) of SGD as the…

We establish existence and uniqueness for the martingale problem associated with a system of degenerate SDE's representing a catalytic branching network. For example, in the hypercyclic case:…

In this paper, we study the problem of escaping from saddle points in smooth nonconvex optimization problems subject to a convex set $\mathcal{C}$. We propose a generic framework that yields convergence to a second-order stationary point of…

We propose a derivative-free saddle-search algorithm designed to locate transition states using only function evaluations. The algorithm employs a nested architecture consisting of an inner eigenvector search and an outer saddle-point…

We consider gradient systems with an increasing potential that depends on a scalar parameter. As the parameter is varied, critical points of the potential can be eliminated or created through saddle-node bifurcations causing the system to…

This work shows that applying Gradient Descent (GD) with a fixed step size to minimize a (possibly nonconvex) quadratic function is equivalent to running the Power Method (PM) on the gradients. The connection between GD with a fixed step…

Recently, the problem of local minima in very high dimensional non-convex optimization has been challenged and the problem of saddle points has been introduced. This paper introduces a dynamic type of normalization that forces the system to…

For continuous \gamma, g:[0,1]\to(0,\infty), consider the degenerate stochastic differential equation dX_t=[1-|X_t|^2]^{1/2}\gamma(|X_t|) dB_t-g(|X_t|)X_t dt in the closed unit ball of R^n. We introduce a new idea to show pathwise…

This paper considers extreme values attained by a centered, multidimensional Gaussian process $X(t)= (X_1(t),\ldots,X_n(t))$ minus drift $d(t)=(d_1(t),\ldots,d_n(t))$, on an arbitrary set $T$. Under mild regularity conditions, we establish…