Related papers: On Convergence Lemma and Convergence Stability for…
This paper discusses a general and useful stability principle which, roughly speaking, says that given a uniformly continuous function defined on an arbitrary metric space, if the function is bounded on the constraint set and we slightly…
We extend to $p$-uniformly convex spaces tools from the analysis of fixed point iterations in linear spaces. This study is restricted to an appropriate generalization of single-valued, pointwise $\alpha$-averaged mappings. Our main…
A lexicographic maximum of a set $X \subseteq \mathbb{R}^n$ is a vector in $X$ whose smallest component is as large as possible, and subject to that requirement, whose second smallest component is as large as possible, and so on for the…
Minimizing a smooth function f on a closed subset C leads to different notions of stationarity: Fr{\'e}chet stationarity, which carries a strong variational meaning, and criticality, which is defined through a closure process and involves…
We prove that the smallest minimizer s(f) of a real convex function f is less than or equal to a real point x if and only if the right derivative of f at x is non-negative. Similarly, the largest minimizer t(f) is greater or equal to x if…
Loss functions with non-isolated minima have emerged in several machine learning problems, creating a gap between theory and practice. In this paper, we formulate a new type of local convexity condition that is suitable to describe the…
This paper considers constrained stochastic nonsmooth minimax optimization problem of the form…
Adam is a popular variant of stochastic gradient descent for finding a local minimizer of a function. In the constant stepsize regime, assuming that the objective function is differentiable and non-convex, we establish the convergence in…
Convergence to a saddle point for convex-concave functions has been studied for decades, while recent years has seen a surge of interest in non-convex (zero-sum) smooth games, motivated by their recent wide applications. It remains an…
We introduce a weak asymptotic version of nonlinear contraction, termed \emph{asymptotic pointwise contraction}. For a mapping on a metric space, this notion requires the existence of a sequence of functions that dominate the distances…
We provide novel theoretical results regarding local optima of regularized $M$-estimators, allowing for nonconvexity in both loss and penalty functions. Under restricted strong convexity on the loss and suitable regularity conditions on the…
Let $C$ be a convex subset of a locally convex space. We provide optimal approximate fixed point results for sequentially continuous maps $f\colon C\to\bar{C}$. First we prove that if $f(C)$ is totally bounded, then it has an approximate…
In this paper, we investigate the non-asymptotic stationary convergence behavior of Stochastic Mirror Descent (SMD) for nonconvex optimization. We focus on a general class of nonconvex nonsmooth stochastic optimization problems, in which…
We address the problem of computing stationary points for non-smooth, non-convex optimization problems. While this topic is well studied in the smooth setting, fewer algorithmic and theoretical results exist for the non-smooth case. Within…
Caputo fractional (with power-law kernels) and fractional (delta) difference maps belong to a more widely defined class of generalized fractional maps, which are discrete convolutions with some power-law-like functions. The conditions of…
Let $f$ be a function on a bounded domain $\Omega \subseteq \mathbb{R}^n$ and $\delta$ be a positive function on $\Omega$ such that $B(x,\delta(x))\subseteq \Omega$. Let $\sigma(f)(x)$ be the average of $f$ over the ball $B(x,\delta(x))$.…
Given a nonconvex function that is an average of $n$ smooth functions, we design stochastic first-order methods to find its approximate stationary points. The convergence of our new methods depends on the smallest (negative) eigenvalue…
In this paper, we study the convergence properties of the Stochastic Gradient Descent (SGD) method for finding a stationary point of a given objective function $J(\cdot)$. The objective function is not required to be convex. Rather, our…
Assume that $X$ is a Banach space of measurable functions for which Koml\'os' Theorem holds. We associate to any closed convex bounded subset $C$ of $X$ a coefficient $t(C)$ which attains its minimum value when $C$ is closed for the…
We show that, for a fixed order $\gamma\geq 1$, each local minimizer of a rather general nonsmooth optimization problem in Euclidean spaces is either M-stationary in the classical sense (corresponding to stationarity of order $1$),…