Related papers: DE-Sinc approximation for unilateral rapidly decre…
The aim of this paper is to apply an original computation method due to Malesevic and Makragic [5] to the problem of approximating some trigonometric functions. Inequalities of Wilker-Cusa-Huygens are discussed, but the method can be…
In this article we recover the distribution function (and possible density) of an arbitrary random variable that is subject to an additive measurement error. This problem is also known as deconvolution and has a long tradition in…
This paper proposes a thorough theoretical analysis of Stochastic Gradient Descent (SGD) with non-increasing step sizes. First, we show that the recursion defining SGD can be provably approximated by solutions of a time inhomogeneous…
In the era of big data, optimizing large scale machine learning problems becomes a challenging task and draws significant attention. Asynchronous optimization algorithms come out as a promising solution. Recently, decoupled asynchronous…
Submodular function minimization is a fundamental optimization problem that arises in several applications in machine learning and computer vision. The problem is known to be solvable in polynomial time, but general purpose algorithms have…
We study the convergence of Stochastic Gradient Descent (SGD) for strongly convex objective functions. We prove for all $t$ a lower bound on the expected convergence rate after the $t$-th SGD iteration; the lower bound is over all possible…
FOCal Underdetermined System Solver (FOCUSS) is a powerful tool for sparse representation and underdetermined inverse problems, which is extremely easy to implement. In this paper, we give a comprehensive convergence analysis on the FOCUSS…
We propose a new method of adaptive piecewise approximation based on Sinc points for ordinary differential equations. The adaptive method is a piecewise collocation method which utilizes Poly-Sinc interpolation to reach a preset level of…
We consider a variable metric and inexact version of the FISTA-type algorithm considered in (Chambolle, Pock, 2016, Calatroni, Chambolle, 2019) for the minimization of the sum of two (possibly strongly) convex functions. The proposed…
This paper presents a family of rapidly convergent summation formulas for various finite sums of analytic functions. These summation formulas are obtained by applying a series acceleration transformation involving Stirling numbers of the…
Gradient descent is a simple and widely used optimization method for machine learning. For homogeneous linear classifiers applied to separable data, gradient descent has been shown to converge to the maximal margin (or equivalently, the…
We present a systematic method for computing explicit approximations to martingale representations for a large class of Brownian functionals. The approximations are obtained by obtained by computing a directional derivative of the weak…
We investigate the strong convergence properties of a proximal-gradient inertial algorithm with two Tikhonov regularization terms in connection to the minimization problem of the sum of a convex lower semi-continuous function $f$ and a…
Diffusion approximation provides weak approximation for stochastic gradient descent algorithms in a finite time horizon. In this paper, we introduce new tools motivated by the backward error analysis of numerical stochastic differential…
The problem of reconstructing functions from their asymptotic expansions in powers of a small variable is addressed by deriving a novel type of approximants. The derivation is based on the self-similar approximation theory, which presents…
Consider the problem of minimizing functions that are Lipschitz and strongly convex, but not necessarily differentiable. We prove that after $T$ steps of stochastic gradient descent, the error of the final iterate is $O(\log(T)/T)$ with…
We consider upper bounds for the approximation error E|g(X)-g(\hat X)|^p, where X and \hat X are random variables such that \hat X is an approximation of X in the L_p-norm, and the function g belongs to certain function classes, which…
The diffusion approximation of stochastic gradient descent (SGD) in current literature is only valid on a finite time interval. In this paper, we establish the uniform-in-time diffusion approximation of SGD, by only assuming that the…
Stochastic approximation (SA) is a classical approach for stochastic convex optimization. Previous studies have demonstrated that the convergence rate of SA can be improved by introducing either smoothness or strong convexity condition. In…
A fast convergence in a fixed-time of solutions of nonlinear dynamical systems, for which special requirements are satisfied on the derivative of a quadratic function calculated along the solutions of the system, is proposed. The conditions…