Related papers: The Subdifferential Descent Method in a Nonsmooth …
Stochastic gradient descent (SGD) method is popular for solving non-convex optimization problems in machine learning. This work investigates SGD from a viewpoint of graduated optimization, which is a widely applied approach for non-convex…
Automatic differentiation, as implemented today, does not have a simple mathematical model adapted to the needs of modern machine learning. In this work we articulate the relationships between differentiation of programs as implemented in…
There has recently been considerable interest in using a nonstandard piecewise approximation to formulate fractional order differential equations as difference equations that describe the same dynamical behaviour and are more amenable to a…
The framework of differential inclusions encompasses modern optimal control and the calculus of variations. Necessary optimality conditions in the literature identify potentially optimal paths, but do not show how to perturb paths to…
Inspired by the recent paper (L. Ying, Mirror descent algorithms for minimizing interacting free energy, Journal of Scientific Computing, 84 (2020), pp. 1-14),we explore the relationship between the mirror descent and the variable metric…
It is well-known that the reparameterisation gradient estimator, which exhibits low variance in practice, is biased for non-differentiable models. This may compromise correctness of gradient-based optimisation methods such as stochastic…
In this work, a class of non-linear weakly singular fractional integro-differential equations is considered, and we first prove existence, uniqueness, and smoothness properties of the solution under certain assumptions on the given data. We…
We propose a novel stochastic gradient descent method for solving linear least squares problems with partially observed data. Our method uses submatrices indexed by a randomly selected pair of row and column index sets to update the iterate…
We develop an efficient stochastic variance reduced gradient descent algorithm to solve the affine rank minimization problem consists of finding a matrix of minimum rank from linear measurements. The proposed algorithm as a stochastic…
We propose an implicit iterative algorithm for an exact penalty method arising from inequality constrained optimization problems. A rapidly convergent fixed point method is developed for a regularized penalty functional. The applicability…
We study the convergence of the gradient descent method for solving ill-posed problems where the solution is characterized as a global minimum of a differentiable functional in a Hilbert space. The classical least-squares functional for…
In this paper we demonstrate that the numerical method of steepest descent fails when applied in a straight forward fashion to the most commonly occurring highly oscillatory integrals in scattering theory. Through a polar change of…
A number of important results of studying large deformations of hyper-elastic shells are obtained using discrete methods of mathematical physics. In the present paper, using the variational method for solving nonlinear boundary problems of…
A systematic digital-discrete method for obtaining continuous functions with smoothness to a certain order (C^(n)) from sample data is designed. This method is based on gradually varied functions and the classical finite difference method.…
Formulas for stable differentiation of piecewise-smooth functions are given. The data are noisy values of these functions. The locations of discontinuity points and the sizes of the jumps across these points are not assumed known, but found…
Fractional Pontryagin's systems emerge in the study of a class of fractional optimal control problems but they are not resolvable in most cases. In this paper, we suggest a numerical approach for these fractional systems. Precisely, we…
Dynamic Mode Decomposition (DMD) has emerged as a powerful tool for analyzing the dynamics of non-linear systems from experimental datasets. Recently, several attempts have extended DMD to the context of low-rank approximations. This…
In this paper, we present a unified analysis of methods for such a wide class of problems as variational inequalities, which includes minimization problems and saddle point problems. We develop our analysis on the modified Extra-Gradient…
In this work, we investigate the idea of variance reduction by studying its properties with general adaptive mirror descent algorithms in nonsmooth nonconvex finite-sum optimization problems. We propose a simple yet generalized framework…
This paper presents a directional proximal point method (DPPM) to derive the minimum of any C1-smooth function f. The proposed method requires a function persistent a local convex segment along the descent direction at any non-critical…