Related papers: One-dimensional System Arising in Stochastic Gradi…
We provide larger step-size restrictions for which gradient descent based algorithms (almost surely) avoid strict saddle points. In particular, consider a twice differentiable (non-convex) objective function whose gradient has Lipschitz…
We study the weak limits of solutions to SDEs \[dX_n(t)=a_n\bigl(X_n(t)\bigr)\,dt+dW(t),\] where the sequence $\{a_n\}$ converges in some sense to $(c_- 1\mkern-4.5mu\mathrm{l}_{x<0}+c_+ 1\mkern-4.5mu\mathrm{l}_{x>0})/x+\gamma\delta_0$.…
We study the saddlepoint approximation (SPA) for sums of $n$ i.i.d. random vectors $X_i\in\mathbb R^d$ in growing dimensions. SPA provides highly accurate approximations to probability densities and distribution functions via the moment…
In this paper we investigate the convergence of a recently popular class of first-order primal-dual algorithms for saddle point problems under the presence of errors occurring in the proximal maps and gradients. We study several types of…
This paper considers continuously differentiable functions of two vector variables that have (possibly a continuum of) min-max saddle points. We study the asymptotic convergence properties of the associated saddle-point dynamics…
We study the stochastic differential equation $dX_t = A(X_{t-}) \, dZ_t$, $ X_0 = x$, where $Z_t = (Z_t^{(1)},\ldots,Z_t^{(d)})^T$ and $Z_t^{(1)}, \ldots, Z_t^{(d)}$ are independent one-dimensional L{\'e}vy processes with characteristic…
In this paper we propose a primal-dual proximal extragradient algorithm to solve the generalized Dantzig selector (GDS) estimation problem, based on a new convex-concave saddle-point (SP) reformulation. Our new formulation makes it possible…
Nesterov's accelerated gradient descent (AGD), an instance of the general family of "momentum methods", provably achieves faster convergence rate than gradient descent (GD) in the convex setting. However, whether these methods are superior…
Let $d \ge 2$. In this paper, we study weak solutions for the following type of stochastic differential equation \[ dX_{t}=dS_{t}+b(s+t, X_{t})dt, \quad X_{0}=x, \] where $(s,x)\in \mathbb{R}_+ \times \mathbb{R}^{d}$ is the initial starting…
We analyze the stochastic proximal subgradient descent in the case where the objective functions are path differentiable and verify a Sard-type condition. While the accumulation set may not be reduced to unique point, we show that the time…
Inverse problems in scientific computing often require optimization over infinite-dimensional Hilbert spaces. A commonly used solver in such settings is stochastic gradient descent (SGD), where gradients are approximated using randomly…
We consider a dynamical system undergoing a saddle-node bifurcation with an explicitly time dependent parameter~$p(t)$. The combined dynamics can be considered as a dynamical systems where $p$ is a slowly evolving parameter. Here, we…
We study several one dimensional step flow models. Numerical simulations show that the slope of the profile exhibits scaling in all cases. We apply a scaling ansatz to the various step flow models and investigate their long time evolution.…
This paper addresses a continuous-time risk-minimizing two-player zero-sum stochastic differential game (SDG), in which each player aims to minimize its probability of failure. Failure occurs in the event when the state of the game enters…
Stochastic dynamical systems allow modelling of transitions induced by disturbances, in particular from an attracting equilibrium and crossing the stable manifold of a saddle. In the small-noise limit, the probability of such transitions is…
This paper discusses the first exit and Dirichlet problems of the nonisotropic tempered $\alpha$-stable process $X_t$. The upper bounds of all moments of the first exit position $\left|X_{\tau_D}\right|$ and the first exit time $\tau_D$ are…
Optimizing non-convex functions is of primary importance in the vast majority of machine learning algorithms. Even though many gradient descent based algorithms have been studied, successive convex approximation based algorithms have been…
We consider stochastic strongly-convex-strongly-concave (SCSC) saddle point (SP) problems which frequently arise in applications ranging from distributionally robust learning to game theory and fairness in machine learning. We focus on the…
Given a stochastic dynamical system modelled via stochastic differential equations (SDEs), we evaluate the safety of the system through characterizations of its exit time moments. We lift the (possibly nonlinear) dynamics into the space of…
We show that convex-concave Lipschitz stochastic saddle point problems (also known as stochastic minimax optimization) can be solved under the constraint of $(\epsilon,\delta)$-differential privacy with \emph{strong (primal-dual) gap} rate…