Related papers: Sufficient conditions for instability of the subgr…
We study equilibrium selection for invariant measures of stochastic dynamical systems with constant step size, under persistent noise and minimal moment assumptions, in a general quasi-Feller framework. Such dynamics arise in…
In this paper two properties of recognized interest in variational analysis, known as Lipschitz lower semicontinuity and calmness, are studied with reference to a general class of variational systems, i.e. to solution mappings to…
We consider integral functionals with slow growth and explicit dependence on u of the lagrangian; this includes many relevant examples, as, for instance, in elastoplastic torsion problems or in image restoration problems. Our aim is to…
We study the oracle complexity of nonsmooth nonconvex optimization, with the algorithm assumed to have access only to local function information. It has been shown by Davis, Drusvyatskiy, and Jiang (2023) that for nonsmooth Lipschitz…
A very simple unidimensional function with Lipschitz continuous gradient is constructed such that the ADAM algorithm with constant stepsize, started from the origin, diverges when applied to minimize this function in the absence of noise on…
We study Stochastic Gradient Descent with AdaGrad stepsizes: a popular adaptive (self-tuning) method for first-order stochastic optimization. Despite being well studied, existing analyses of this method suffer from various shortcomings:…
In the present paper, a systematic study is made of quantitative semicontinuity (a.k.a. Lipschitzian) properties of certain multifunctions, which are defined as a solution map associated to a family of parameterized ``split" feasibility…
In this paper we measured the stability of stochastic gradient method (SGM) for learning an approximated Fourier primal support vector machine. The stability of an algorithm is considered by measuring the generalization error in terms of…
Classical optimisation theory guarantees monotonic objective decrease for gradient descent (GD) when employed in a small step size, or ``stable", regime. In contrast, gradient descent on neural networks is frequently performed in a large…
In this paper, we consider two variants of the concept of sharp minimum for mathematical programming problems with quasiconvex objective function and inequality constraints. It investigated the problem of describing a variant of a simple…
Approximate necessary optimality conditions in terms of Fr\'echet subgradients and normals for a rather general optimization problem with a potentially non-Lipschitzian objective function are established with the aid of Ekeland's…
Block-coordinate algorithms are recognized to furnish efficient iterative schemes for addressing large-scale problems, especially when the computation of full derivatives entails substantial memory requirements and computational efforts. In…
Subgradient methods are the natural extension to the non-smooth case of the classical gradient descent for regular convex optimization problems. However, in general, they are characterized by slow convergence rates, and they require…
In this note we prove that on metric measure spaces, functions of least gradient, as well as local minimizers of the area functional (after modification on a set of measure zero) are continuous everywhere outside their jump sets. As a tool,…
Two genres of heuristics that are frequently reported to perform much better on "real-world" instances than in the worst case are greedy algorithms and local search algorithms. In this paper, we systematically study these two types of…
A subgradient method is presented for solving general convex optimization problems, the main requirement being that a strictly-feasible point is known. A feasible sequence of iterates is generated, which converges to within user-specified…
We study the iteration complexity of Lipschitz convex optimization problems satisfying a general error bound. We show that for this class of problems, subgradient descent with either Polyak stepsizes or decaying stepsizes achieves minimax…
The Lipschitz constant is an important quantity that arises in analysing the convergence of gradient-based optimization methods. It is generally unclear how to estimate the Lipschitz constant of a complex model. Thus, this paper studies an…
Drori and Teboulle [4] conjectured that the minimax optimal constant stepsize for N steps of gradient descent is given by the stepsize that balances performance on Huber and quadratic objective functions. This was numerically supported by…
This paper considers stochastic weakly convex optimization without the standard Lipschitz continuity assumption. Based on new adaptive regularization (stepsize) strategies, we show that a wide class of stochastic algorithms, including the…