Related papers: On Convergence Lemma and Convergence Stability for…
We study the stable behaviour of discrete dynamical systems where the map is convex and monotone with respect to the standard positive cone. The notion of tangential stability for fixed points and periodic points is introduced, which is…
Many problems in machine learning can be formulated as optimizing a convex functional over a vector space of measures. This paper studies the convergence of the mirror descent algorithm in this infinite-dimensional setting. Defining Bregman…
Approximations of optimization problems arise in computational procedures and sensitivity analysis. The resulting effect on solutions can be significant, with even small approximations of components of a problem translating into large…
We study projective functions. We prove that projective functions generalise lower and upper-semianalytic ones while being stable by composition and difference. We show that the class of projective functions is closed under sums,…
Stochastic approximation (SA) is a classical approach for stochastic convex optimization. Previous studies have demonstrated that the convergence rate of SA can be improved by introducing either smoothness or strong convexity condition. In…
In this work we examine the stability of some classes of integrals, and in particular with respect to homogenization. The prototypical case is the homogenization of quadratic energies with periodic coefficients perturbed by a term vanishing…
Sharpness-aware minimization (SAM) is a recently proposed training method that seeks to find flat minima in deep learning, resulting in state-of-the-art performance across various domains. Instead of minimizing the loss of the current…
We study when the \emph{optimization curve} of first-order methods -- the sequence \${f(x\_n)}*{n\ge0}\$ produced by constant-stepsize iterations -- is convex, equivalently when the forward differences \$f(x\_n)-f(x*{n+1})\$ are…
We develop a rigorous framework for global non-convex optimization by reformulating the minimization problem as a discounted infinite-horizon optimal control problem. For non-convex, continuous, and possibly non-smooth objective functions…
Let $S$ be a right reversible semitopological semigroup, and let $\operatorname{LUC}(S)$ be the space of left uniformly continuous functions on $S$. Suppose that $\operatorname{LUC}(S)$ has a left invariant mean. Let $K$ be a weakly compact…
In this paper, we propose a successive pseudo-convex approximation algorithm to efficiently compute stationary points for a large class of possibly nonconvex optimization problems. The stationary points are obtained by solving a sequence of…
Solving bilevel optimization (BLO) problems to global optimality is generally intractable. A common surrogate is to compute a hyper-stationary point -- a stationary point of the hyper-objective function obtained by minimizing or maximizing…
This paper studies the complexity for finding approximate stationary points of nonconvex-strongly-concave (NC-SC) smooth minimax problems, in both general and averaged smooth finite-sum settings. We establish nontrivial lower complexity…
This paper concerns piecewise-smooth maps on $\mathbb{R}^d$ that are continuous but not differentiable on switching manifolds (where the functional form of the map changes). The stability of fixed points on switching manifolds is…
A Riemannian metric with a local contraction property can be used to prove existence and uniqueness of a periodic orbit and determine a subset of its basin of attraction. While the existence of such a contraction metric is equivalent to the…
We introduce Z-stability, a notion capturing the intuition that if a function f maps a metric space into a normed space and if the norm of f(x) is small, then x is close to a zero of f. Working in Bishop's constructive setting, we first…
We present a fixed point theorem for a class of (potentially) non-monotonic functions over specially structured complete lattices. The theorem has as a special case the Knaster-Tarski fixed point theorem when restricted to the case of…
In deep learning (DL) the instability phenomenon is widespread and well documented, most commonly using the classical measure of stability, the Lipschitz constant. While a small Lipchitz constant is traditionally viewed as guarantying…
Stochastic non-smooth convex optimization constitutes a class of problems in machine learning and operations research. This paper considers minimization of a non-smooth function based on stochastic subgradients. When the function has a…
We show that the solutions to the nonlocal obstacle problems for the nonlocal $-\Delta_p^s$ operator, when the fractional parameter $s\to\sigma$ for $0<\sigma\leq1$, converge to the solution of the corresponding obstacle problem for…