Related papers: Convergence to minima for the continuous version o…

The subgradient method is one of the most fundamental algorithmic schemes for nonsmooth optimization. The existing complexity and convergence results for this method are mainly derived for Lipschitz continuous objective functions. In this…

Given a family of locally Lipschitz vector fields $X(x)=(X_1(x),\dots,X_m(x))$ on $\mathbb{R}^n$, $m\leq n$, we study integral functionals depending on $X$. Using the results in \cite{MPSC1}, we study the convergence of minima, minimizers…

In this note, as a particular case of a more general result, we obtain the following theorem: Let $\Omega\subseteq {\bf R}^n$ be a non-empty bounded open set and let $f:\overline {\Omega}\to {\bf R}^n$ be a continuous function which is…

We first study the fast minimization properties of the trajectories of the second-order evolution equation $$\ddot{x}(t) + \frac{\alpha}{t} \dot{x}(t) + \beta \nabla^2 \Phi (x(t))\dot{x} (t) + \nabla \Phi (x(t)) = 0,$$ where $\Phi:\mathcal…

It was established in [8] that Lipschitz inf-compact functions are uniquely determined by their local slope and critical values. Compactness played a paramount role in this result, ensuring in particular the existence of critical points. We…

We consider the Cauchy problem for the gradient flow \begin{equation} \label{eq:81} \tag{$\star$} u'(t)=-\nabla\phi(u(t)),\quad t\ge 0;\quad u(0)=u_0, \end{equation} generated by a continuously differentiable function $\phi:\mathbb H \to…

This paper studies the asymptotic behavior of the constant step Stochastic Gradient Descent for the minimization of an unknown function F , defined as the expectation of a non convex, non smooth, locally Lipschitz random function. As the…

We provide new insight into a {\em generalized conditional subgradient} algorithm and a {\em generalized mirror descent} algorithm for the convex minimization problem \[ \min_x \; \{f(Ax) + h(x)\}.\] As Bach showed in [{\em SIAM J. Optim.},…

In a real Hilbert space $\mathcal H$, we study the fast convergence properties as $t \to + \infty$ of the trajectories of the second-order evolution equation $$ \ddot{x}(t) + \frac{\alpha}{t} \dot{x}(t) + \nabla \Phi (x(t)) = 0, $$ where…

Convergence of the gradient descent algorithm has been attracting renewed interest due to its utility in deep learning applications. Even as multiple variants of gradient descent were proposed, the assumption that the gradient of the…

In this paper, we consider the Forward--Backward proximal splitting algorithm to minimize the sum of two proper convex functions, one of which having a Lipschitz continuous gradient and the other being partly smooth relative to an active…

We propose a variable metric framework for minimizing the sum of a self-concordant function and a possibly non-smooth convex function, endowed with an easily computable proximal operator. We theoretically establish the convergence of our…

We study the stochastic optimization problem from a continuous-time perspective, with a focus on the Stochastic Gradient Descent with Momentum (SGDM) method. We show that the trajectory of SGDM, despite its \emph{stochastic} nature,…

We prove that every continuous mapping from a separable infinite-dimensional Hilbert space $X$ into $\mathbb{R}^{m}$ can be uniformly approximated by $C^\infty$ smooth mappings {\em with no critical points}. This kind of result can be…

Given a non-convex twice continuously differentiable cost function with Lipschitz continuous gradient, we prove that all of block coordinate gradient descent, block mirror descent and proximal block coordinate descent converge to a local…

We revisit the geometrically decaying step size given a positive inverse condition number, under which a locally Lipschitz function shows linear convergence. The positivity does not require the function to satisfy convexity, weak convexity,…

We investigate the convergence properties of exact and inexact forward-backward algorithms to minimise the sum of two weakly convex functions defined on a Hilbert space, where one has a Lipschitz-continuous gradient. We show that the exact…

We consider minimizing a nonconvex, smooth function $f$ on a Riemannian manifold $\mathcal{M}$. We show that a perturbed version of Riemannian gradient descent algorithm converges to a second-order stationary point (and hence is able to…

We study the convergence rate of the proximal-gradient homotopy algorithm applied to norm-regularized linear least squares problems, for a general class of norms. The homotopy algorithm reduces the regularization parameter in a series of…

In this paper, two main results concerning uniformly continuous retractions are proved. First, an $\alpha$-H\"older retraction from any separable Banach space onto a compact convex subset whose closed linear span is the whole space is…