Related papers: Approximating gradients with continuous piecewise …
We consider a stochastic version of the proximal point algorithm for optimization problems posed on a Hilbert space. A typical application of this is supervised learning. While the method is not new, it has not been extensively analyzed in…
We propose a new numerical scheme for approximating level-sets of Lipschitz multivariate functions which is robust to stochastic noise. The algorithm's main feature is an adaptive grid-based stochastic approximation strategy which…
We propose an adaptive finite element algorithm to approximate solutions of elliptic problems whose forcing data is locally defined and is approximated by regularization (or mollification). We show that the energy error decay is…
We analyze the constant step size subgradient method on nonsmooth, nonconvex functions. We identify geometric assumptions on the objective function under which i) its domain admits a partition (stratification) into smooth manifolds (strata)…
We compute the closest convex piecewise linear-quadratic (PLQ) function with minimal number of pieces to a given univariate piecewise linear-quadratic function. The Euclidean norm is used to measure the distance between functions. First, we…
A sequential piecewise linear programming method is presented where bounded domains of non-convex functions are successively contracted about the solution of a piecewise linear program at each iteration of the algorithm. Although…
Adaptive Finite Element Method (adaptivity) is known to be an effective numerical tool for some ill-posed problems. The key advantage of the adaptivity is the image improvement with local mesh refinements. A rigorous proof of this property…
The paper deals with finite element approximations of elliptic Dirichlet boundary control problems posed on two-dimensional polygonal domains. Error estimates are derived for the approximation of the control and the state variables. Special…
We prove linear convergence of gradient descent to a global optimum for the training of deep residual networks with constant layer width and smooth activation function. We show that if the trained weights, as a function of the layer index,…
In this paper, we analyze several methods for approximating gradients of noisy functions using only function values. These methods include finite differences, linear interpolation, Gaussian smoothing and smoothing on a sphere. The methods…
A common approach to compute distances on continuous surfaces is by considering a discretized polygonal mesh approximating the surface and estimating distances on the polygon. We show that exact geodesic distances restricted to the polygon…
The piecewise-concave function may be used to approximate a wide range of other functions to arbitrary precision over a bounded set. In this short paper, this property is proven for three function classes: (a) the multivariate twice…
We present and analyse a numerical framework for the approximation of nonlinear degenerate elliptic equations of the Stefan or porous medium types. This framework is based on piecewise constant approximations for the functions, which we…
We propose a new stochastic gradient method for optimizing the sum of a finite set of smooth functions, where the sum is strongly convex. While standard stochastic gradient methods converge at sublinear rates for this problem, the proposed…
In this paper we propose a variant of the random coordinate descent method for solving linearly constrained convex optimization problems with composite objective functions. If the smooth part of the objective function has Lipschitz…
We consider the problem of optimising the expected value of a loss functional over a nonlinear model class of functions, assuming that we have only access to realisations of the gradient of the loss. This is a classical task in statistics,…
We describe an approximate dynamic programming approach to compute lower bounds on the optimal value function for a discrete time, continuous space, infinite horizon setting. The approach iteratively constructs a family of lower bounding…
In this paper, we study adaptive finite element approximations in a perturbation framework, which makes use of the existing adaptive finite element analysis of a linear symmetric elliptic problem. We prove the convergence and complexity of…
We propose a new length formula that governs the iterates of the momentum method when minimizing differentiable semialgebraic functions with locally Lipschitz gradients. It enables us to establish local convergence, global convergence, and…
We present a fully iterative adaptive algorithm for the numerical minimization of strongly convex energy functionals in Hilbert spaces. The proposed approach, which we first present in abstract form, generates a hierarchical sequence of…