Related papers: Gradients on Sets

This paper introduces a coordinate descent version of the V\~u-Condat algorithm. By coordinate descent, we mean that only a subset of the coordinates of the primal and dual iterates is updated at each iteration, the other coordinates being…

A framework is presented whereby a general convex conic optimization problem is transformed into an equivalent convex optimization problem whose only constraints are linear equations and whose objective function is Lipschitz continuous.…

The aim of this paper is to prove the exponential convergence, local and global, of Adam algorithm under precise conditions on the parameters, when the objective function lacks differentiability. More precisely, we require Lipschitz…

In this paper we propose a distributed version of a randomized block-coordinate descent method for minimizing the sum of a partially separable smooth convex function and a fully separable non-smooth convex function. Under the assumption of…

We give quantum speedups of several general-purpose numerical optimisation methods for minimising a function $f:\mathbb{R}^n \to \mathbb{R}$. First, we show that many techniques for global optimisation under a Lipschitz constraint can be…

We prove the first convergence guarantees for a subgradient method minimizing a generic Lipschitz function over generic Lipschitz inequality constraints. No smoothness or convexity (or weak convexity) assumptions are made. Instead, we…

We study gradient testing and gradient estimation of smooth functions using only a comparison oracle that, given two points, indicates which one has the larger function value. For any smooth $f\colon\mathbb R^n\to\mathbb R$,…

The classical McShane-Whitney extension theorem for Lipschitz functions is refined by showing that for a closed subset of the domain, it remains valid for any interval of the real line. This result is also extended to the setting of locally…

We use the framework of a type of abstract convexity ($\Phi_{lsc}$-convexity) to investigate properties of lower semicontinuous quadratically minorized functions in Hilbert spaces. A new result, which states that, for every local…

In this paper, we study the regularity of several notions of Lipschitz solutions to the minimal surface system with an emphasis on partial regularity results. These include stationary solutions, integral weak solutions, and viscosity…

We obtain a local estimate for the gradient of solutions to a second-order elliptic equation in divergence form with bounded measurable coefficients that are square-Dini continuous at the single point x=0. In particular, we treat the case…

Based on a result by Taylor, Hendrickx, and Glineur (J. Optim. Theory Appl., 178(2):455--476, 2018) on the attainable convergence rate of gradient descent for smooth and strongly convex functions in terms of function values, an elementary…

We study when the \emph{optimization curve} of first-order methods -- the sequence \${f(x\_n)}*{n\ge0}\$ produced by constant-stepsize iterations -- is convex, equivalently when the forward differences \$f(x\_n)-f(x*{n+1})\$ are…

In a previous paper it was shown that the Forward Euler method applied to differential inclusions where the right-hand side is a Lipschitz continuous set-valued function with uniformly bounded, compact values, converges with rate one. The…

Gradient algorithms are classical in adaptive control and parameter estimation. For instantaneous quadratic cost functions they lead to a linear time-varying dynamic system that converges exponentially under persistence of excitation…

Iteration complexities for optimizing smooth functions with first-order algorithms are typically stated in terms of a global Lipschitz constant of the gradient, and near-optimal results are then achieved using fixed step sizes. But many…

We consider the problem of minimizing a convex objective which is the sum of a smooth part, with Lipschitz continuous gradient, and a nonsmooth part. Inspired by various applications, we focus on the case when the nonsmooth part is a…

This paper deals with local criteria for the convergence to a global minimiser for gradient flow trajectories and their discretisations. To obtain quantitative estimates on the speed of convergence, we consider variations on the classical…

We consider a fractional generalization of Hamiltonian and gradient systems. We use differential forms and exterior derivatives of fractional orders. We derive fractional generalization of Helmholtz conditions for phase space. Examples of…

One of the most important parts of Artificial Neural Networks is minimizing the loss functions which tells us how good or bad our model is. To minimize these losses we need to tune the weights and biases. Also to calculate the minimum value…