Related papers: On the diameter of subgradient sequences in o-mini…

Composite optimization problems, where the sum of a smooth and a merely lower semicontinuous function has to be minimized, are often tackled numerically by means of proximal gradient methods as soon as the lower semicontinuous part of the…

In this paper, we study the proximal gradient algorithm with extrapolation for minimizing the sum of a Lipschitz differentiable function and a proper closed convex function. Under the error bound condition used in [19] for analyzing the…

The paper addresses the study and applications of a broad class of extended-real-valued functions, known as optimal value or marginal functions, which are frequently appeared in variational analysis, parametric optimization, and a variety…

We study the variable metric forward-backward splitting algorithm for convex minimization problems without the standard assumption of the Lipschitz continuity of the gradient. In this setting, we prove that, by requiring only mild…

We investigate the stochastic gradient descent (SGD) method where the step size lies within a banded region instead of being given by a fixed formula. The optimal convergence rate under mild conditions and large initial step size is proved.…

In this paper, we revisit the Mordukhovich's subdifferential criterion for Lipschitz continuity of nonsmooth functions and coderivative criterion for the Aubin/Lipschitz-like property of set-valued mappings in finite dimensions. The…

We prove various theorems on approximation using polynomials with integer coefficients in the Bernstein basis of any given order. In the extreme, we draw the coefficients from $\{ \pm 1\}$ only. A basic case of our results states that for…

Given a non-convex twice continuously differentiable cost function with Lipschitz continuous gradient, we prove that all of block coordinate gradient descent, block mirror descent and proximal block coordinate descent converge to a local…

In this paper, we study classes of discrete convex functions: submodular functions on modular semilattices and L-convex functions on oriented modular graphs. They were introduced by the author in complexity classification of minimum…

We study the local Lipschitz one subsets of a finite dimensional space, that is, sets for which there exists a continuous function whose local Lipschitz derivative is the characteristic function of said set. We give a characterization of a…

The aim of this paper is to deepen the convergence analysis of the scaled gradient projection (SGP) method, proposed by Bonettini et al. in a recent paper for constrained smooth optimization. The main feature of SGP is the presence of a…

We prove lower bounds on the error incurred when approximating any oscillating function using piecewise polynomial spaces. The estimates are explicit in the polynomial degree and have optimal dependence on the meshwidth and frequency when…

In this paper we present a new method for solving optimization problems involving the sum of two proper, convex, lower semicontinuous functions, one of which has Lipschitz continuous gradient. The proposed method has a hybrid nature that…

A subspace $H$ of a rearrangement invariant space $X$ on $[0,1]$ is strongly embedded in $X$ if, in $H$, convergence in $X$-norm is equivalent to convergence in measure. We obtain necessary and sufficient conditions on an Orlicz function…

We consider gradient descent with constant stepsizes and derive exact worst-case convergence rates on the minimum gradient norm of the iterates. Our analysis covers all possible stepsizes and arbitrary upper/lower bounds on the curvature of…

We analyze the accuracy of the discrete least-squares approximation of a function $u$ in multivariate polynomial spaces $\mathbb{P}_\Lambda:={\rm span} \{y\mapsto y^\nu \,: \, \nu\in \Lambda\}$ with $\Lambda\subset \mathbb{N}_0^d$ over the…

Classical optimisation theory guarantees monotonic objective decrease for gradient descent (GD) when employed in a small step size, or ``stable", regime. In contrast, gradient descent on neural networks is frequently performed in a large…

Iteration complexities for optimizing smooth functions with first-order algorithms are typically stated in terms of a global Lipschitz constant of the gradient, and near-optimal results are then achieved using fixed step sizes. But many…

In this paper we propose a variant of the random coordinate descent method for solving linearly constrained convex optimization problems with composite objective functions. If the smooth part of the objective function has Lipschitz…

In Euclidean space of dimension 2 or 3, we study a minimum time problem associated with a system of real-analytic vector fields satisfying H\"ormander's bracket generating condition, where the target is a nonempty closed set. We show that,…