Related papers: Smooth, globally Polyak-{\L}ojasiewicz functions a…
Let $F(x) := (f_{ij}(x))_{i=1,\ldots,p; j=1,\ldots,q},$ be a ($p\times q$)-real polynomial matrix and let $f(x)$ be the smallest singular value function of $F(x).$ In this paper, we first give the following {\em nonsmooth} version of \L…
The classical Lojasiewicz inequality and its extensions for partial differential equation problems (Simon) and to o-minimal structures (Kurdyka) have a considerable impact on the analysis of gradient-like methods and related problems:…
This paper focuses on stochastic methods for solving smooth non-convex strongly-concave min-max problems, which have received increasing attention due to their potential applications in deep learning (e.g., deep AUC maximization,…
Let $f$ be an isolated singularity at the origin of $\mathbb{C}^n$. One of many invariants that can be associated with $f$ is its {\L}ojasiewicz exponent $\mathcal{L}_0 (f)$, which measures, to some extent, the topology of $f$. We give, for…
We develop efficient algorithms for optimizing piecewise smooth (PWS) functions where the underlying partition of the domain into smooth pieces is \emph{unknown}. For PWS functions satisfying a quadratic growth (QG) condition, we propose a…
A basic version of the P\'olya-Szeg\H{o} inequality states that if $\Phi$ is a Young function, the $\Phi$-Dirichlet energy -- the integral of $\Phi(\|\nabla f\|)$ -- of a suitable function $f\in \mathcal{V}(\mathbb{R}^n)$, the class of…
In this paper we propose a generalized condition for a sharp minimum, somewhat similar to the inexact oracle proposed recently by Devolder-Glineur-Nesterov. The proposed approach makes it possible to extend the class of applicability of…
We consider a composite optimization problem where the sum of a continuously differentiable and a merely lower semicontinuous function has to be minimized. The proximal gradient algorithm is the classical method for solving such a problem…
We consider an extension of the Newton-MR algorithm for nonconvex unconstrained optimization to the settings where Hessian information is approximated. Under a particular noise model on the Hessian matrix, we investigate the iteration and…
An iterative optimization method applied to a function $f$ on $\mathbb{R}^n$ will produce a sequence of arguments $\{\mathbf{x}_k\}_{k \in \mathbb{N}}$; this sequence is often constrained such that $\{f(\mathbf{x}_k)\}_{k \in \mathbb{N}}$…
Potential functions in highly pertinent applications, such as deep learning in over-parameterized regime, are empirically observed to admit non-isolated minima. To understand the convergence behavior of stochastic dynamics in such…
Potential functions in highly pertinent applications, such as deep learning in over-parameterized regime, are empirically observed to admit non-isolated minima. To understand the convergence behavior of stochastic dynamics in such…
For any $M, n \geq 2$ and any open set $\Omega \subset \mathbb{R}^n$ we find a smooth, strongly polyconvex function $F\colon \mathbb{R}^{M\times n}\to \mathbb{R}$ and a Lipschitz map $u\colon \mathbb{R}^n \to \mathbb{R}^M$ that is a weak…
We propose a proximal algorithm for minimizing objective functions consisting of three summands: the composition of a nonsmooth function with a linear operator, another nonsmooth function, each of the nonsmooth summands depending on an…
Randomized smoothing is a widely adopted technique for optimizing nonsmooth objective functions. However, its efficiency analysis typically relies on global Lipschitz continuity, a condition rarely met in practical applications. To address…
We provide a comprehensive study of the convergence of the forward-backward algorithm under suitable geometric conditions, such as conditioning or {\L}ojasiewicz properties. These geometrical notions are usually local by nature, and may…
We propose a manifold AdaGrad-Norm method (\textsc{MAdaGrad}), which extends the norm version of AdaGrad (AdaGrad-Norm) to Riemannian optimization. In contrast to line-search schemes, which may require several exponential map computations…
The main problem considered in the present paper is to single out classes of convex sets, whose convexity property is preserved under nonlinear smooth transformations. Extending an approach due to B.T. Polyak, the present study focusses on…
Stochastic gradient methods with momentum are widely used in applications and at the core of optimization subroutines in many popular machine learning libraries. However, their sample complexities have not been obtained for problems beyond…
In this paper we present necessary and sufficient conditions (in terms of {\L}ojasiewicz inequalities) for the stability of local minimum points in smooth unconstrained optimization. In particular, we derive a sufficient condition for which…