Related papers: Gradients on Sets

Convergence of the gradient descent algorithm has been attracting renewed interest due to its utility in deep learning applications. Even as multiple variants of gradient descent were proposed, the assumption that the gradient of the…

We analyze nonlinearly preconditioned gradient methods for solving smooth minimization problems. We introduce a generalized smoothness property, based on the notion of abstract convexity, that is broader than Lipschitz smoothness and…

In this paper, we develop new first-order method for composite non-convex minimization problems with simple constraints and inexact oracle. The objective function is given as a sum of "`hard"', possibly non-convex part, and "`simple"'…

Theoretical estimates of the convergence rate of many well-known gradient-type optimization methods are based on quadratic interpolation, provided that the Lipschitz condition for the gradient is satisfied. In this article we obtain a…

We establish existence of steepest descent curves emanating from almost every point of a regular locally Lipschitz quasiconvex functions, where regularity means that the sweeping process flow induced by the sublevel sets is reversible. We…

This paper studies proximal gradient iterations for solving simple bilevel optimization problems where both the upper and the lower level cost functions are split as the sum of differentiable and (possibly nonsmooth) proximable functions.…

The classical Mountain Pass Lemma of Ambrosetti-Rabinowitz has been studied, extended and modified in several directions. Notable examples would certainly include the generalization to locally Lipschitz functionals by K.C. Chang, analyzing…

We first study Clarke's tangent cones at infinity to unbounded subsets of $\mathbb{R}^n.$ We prove that these cones are closed convex and show a characterization of their interiors. We then study subgradients at infinity for extended real…

Regularization is a widely recognized technique in mathematical optimization. It can be used to smooth out objective functions, refine the feasible solution set, or prevent overfitting in machine learning models. Due to its simplicity and…

The usual approach to developing and analyzing first-order methods for smooth convex optimization assumes that the gradient of the objective function is uniformly smooth with some Lipschitz constant $L$. However, in many settings the…

Decentralized optimization has become a fundamental tool for large-scale learning systems; however, most existing methods rely on the classical Lipschitz smoothness assumption, which is often violated in problems with rapidly varying…

In this paper, locally Lipschitz, regular functions are utilized to identify and remove infeasible directions from set-valued maps that define differential inclusions. The resulting reduced set-valued map is point-wise smaller (in the sense…

Global minimization is a fundamental challenge in optimization, especially in machine learning, where finding the global minimum of a function directly impacts model performance and convergence. This article introduces a novel optimization…

We consider the minimization of non-convex quadratic forms regularized by a cubic term, which exhibit multiple saddle points and poor local minima. Nonetheless, we prove that, under mild assumptions, gradient descent approximates the…

We construct a differentiable locally Lipschitz function $f$ in $\mathbb{R}^{N}$ with the property that for every convex body $K\subset \mathbb{R}^N$ there exists $\bar x \in \mathbb{R}^N$ such that $K$ coincides with the set $\partial_L…

We provide new gradient-based methods for efficiently solving a broad class of ill-conditioned optimization problems. We consider the problem of minimizing a function $f : \mathbb{R}^d \rightarrow \mathbb{R}$ which is implicitly…

Optimization problems with rank constraints arise in many applications, including matrix regression, structured PCA, matrix completion and matrix decomposition problems. An attractive heuristic for solving such problems is to factorize the…

Motivated by applications to stochastic programming, we introduce and study the expected-integral functionals, which are mappings given in an integral form depending on two variables, the first a finite dimensional decision vector and the…

We investigate smooth approximations of functions, with prescribed gradient behavior on a distinguished stratified subset of the domain. As an application, we outline how our results yield important consequences for a recently introduced…

We show that adaptive proximal gradient methods for convex problems are not restricted to traditional Lipschitzian assumptions. Our analysis reveals that a class of linesearch-free methods is still convergent under mere local H\"older…