Related papers: On Convergence to Essential Singularities

Let $X\subset \mathbb{R}^n$ be a compact semialgebraic set and let $f:X\to \mathbb{R}$ be a nonzero Nash function. We give a Solern\'o and D'Acunto-Kurdyka type estimation of the exponent $\varrho\in[0,1)$ in the {\L}ojasiewicz gradient…

Recursive stochastic algorithms have gained significant attention in the recent past due to data driven applications. Examples include stochastic gradient descent for solving large-scale optimization problems and empirical dynamic…

We present a new algorithm for solving optimization problems with objective functions that are the sum of a smooth function and a (potentially) nonsmooth regularization function, and nonlinear equality constraints. The algorithm may be…

The Polyak-{\L}ojasiewicz (P{\L}) inequality extends the favorable optimization properties of strongly convex functions to a broader class of functions. In this paper, we prove a theorem (also obtained by Criscitiello, Rebjock and Boumal in…

In this paper, we consider the convergence of an abstract inexact nonconvex and nonsmooth algorithm. We promise a pseudo sufficient descent condition and a pseudo relative error condition, which are both related to an auxiliary sequence,…

This paper focuses on the minimization of a sum of a twice continuously differentiable function $f$ and a nonsmooth convex function. An inexact regularized proximal Newton method is proposed by an approximation to the Hessian of $f$…

Let $F(x) := (f_{ij}(x))_{i=1,\ldots,p; j=1,\ldots,q},$ be a ($p\times q$)-real polynomial matrix and let $f(x)$ be the smallest singular value function of $F(x).$ In this paper, we first give the following {\em nonsmooth} version of \L…

The {\L}ojasiewicz inequality characterizes objective-value convergence along gradient flows and, in special cases, yields exponential decay of the cost. However, such results do not directly give rates of convergence in the state. In this…

Nesterov's accelerated gradient method for minimizing a smooth strongly convex function $f$ is known to reduce $f(\x_k)-f(\x^*)$ by a factor of $\eps\in(0,1)$ after $k\ge O(\sqrt{L/\ell}\log(1/\eps))$ iterations, where $\ell,L$ are the two…

In this work, we establish that Nesterov's accelerated gradient method, applied to $C^2$ functions satisfying the Polyak--{\L}ojasiewicz inequality around local minimizers, achieves the optimal local linear convergence rate…

Let $(X_{n,i})_{1\le i\le n,n\in\mathbb{N}}$ be a triangular array of row-wise stationary $\mathbb{R}^d$-valued random variables. We use a "blocks method" to define clusters of extreme values: the rows of $(X_{n,i})$ are divided into $m_n$…

We consider the dynamical system \begin{equation*}\left\{ \begin{array}{ll} v(t)\in\partial\phi(x(t))\\ \lambda\dot x(t) + \dot v(t) + v(t) + \nabla \psi(x(t))=0, \end{array}\right.\end{equation*} where $\phi:\R^n\to\R\cup\{+\infty\}$ is a…

Stochastic nonconvex-concave min-max saddle point problems appear in many machine learning and control problems including distributionally robust optimization, generative adversarial networks, and adversarial learning. In this paper, we…

We study the convergence rate of gradient-based local search methods for solving low-rank matrix recovery problems with general objectives in both symmetric and asymmetric cases, under the assumption of the restricted isometry property.…

In this paper, we study the convergence properties of the Stochastic Gradient Descent (SGD) method for finding a stationary point of a given objective function $J(\cdot)$. The objective function is not required to be convex. Rather, our…

We study the convergence properties of the original and away-step Frank-Wolfe algorithms for linearly constrained stochastic optimization assuming the availability of unbiased objective function gradient estimates. The objective function is…

We propose and analyze a randomized zeroth-order approach based on approximating the exact gradient byfinite differences computed in a set of orthogonal random directions that changes with each iteration. A number ofpreviously proposed…

We propose a derivative-free trust-region method based on finite-difference gradient approximations for smooth optimization problems with convex constraints. The proposed method does not require computing an approximate stationarity…

We propose a proximal algorithm for minimizing objective functions consisting of three summands: the composition of a nonsmooth function with a linear operator, another nonsmooth function, each of the nonsmooth summands depending on an…

We analyze the complexity of biased stochastic gradient methods (SGD), where individual updates are corrupted by deterministic, i.e. biased error terms. We derive convergence results for smooth (non-convex) functions and give improved rates…