Related papers: Variable stepsize SDIMSIMs for ordinary differenti…
High order methods are often desired for the evolution of ordinary differential equations, in particular those arising from the semi-discretization of partial differential equations. In prior work in we investigated the interplay between…
We introduce a new adaptive step-size strategy for convex optimization with stochastic gradient that exploits the local geometry of the objective function only by means of a first-order stochastic oracle and without any hyper-parameter…
The paper is devoted to deriving novel second-order necessary and sufficient optimality conditions for local minimizers in rather general classes of nonsmooth unconstrained and constrained optimization problems in finite-dimensional spaces.…
We consider difference schemes for nonlinear time fractional Klein-Gordon type equations in this paper. A linearized scheme is proposed to solve the problem. As a result, iterative method need not be employed. One of the main difficulties…
Component-wise gradient boosting algorithms are popular for their intrinsic variable selection and implicit regularization, which can be especially beneficial for very flexible model classes. When estimating generalized additive models for…
We present an adaptive arbitrary-order accurate time-stepping numerical scheme for the flow of vesicles suspended in Stokesian fluids. Our scheme can be summarized as an approximate implicit spectral deferred correction (SDC) method.…
We present a class of nonconforming virtual element methods for general fourth order partial differential equations in two dimensions. We develop a generic approach for constructing the necessary projection operators and virtual element…
We present a new line-based discontinuous Galerkin (DG) discretization scheme for first- and second-order systems of partial differential equations. The scheme is based on fully unstructured meshes of quadrilateral or hexahedral elements,…
Using geometric methods for linearizing systems of second order cubically semi-linear ordinary differential equations and third order quintically semi-linear ordinary differential equations, we extend to the fourth order by differentiating…
The Barzilai-Borwein (BB) gradient method is efficient for solving large-scale unconstrained problems to the modest accuracy and has a great advantage of being easily extended to solve a wide class of constrained optimization problems. In…
We propose a stochastic conditional gradient method (CGM) for minimizing convex finite-sum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this template either suffer from slow convergence rates, or…
The stochastic Polyak step size (SPS) has proven to be a promising choice for stochastic gradient descent (SGD), delivering competitive performance relative to state-of-the-art methods on smooth convex and non-convex optimization problems,…
In this paper, two low-complexity adaptive step size algorithms are investigated for blind adaptive beamforming. Both of them are used in a stochastic gradient (SG) algorithm, which employs the constrained constant modulus (CCM) criterion…
Nonlinear stochastic motion presents significant challenges for Bayesian particle tracking. To address this challenge, this paper proposes a framework to construct an invertible transformation that maps the nonlinear state-space model (SSM)…
We analyze the variance of stochastic gradients along negative curvature directions in certain non-convex machine learning models and show that stochastic gradients exhibit a strong component along these directions. Furthermore, we show…
In the report, we propose a family of variable time-stepping ensemble algorithms for solving multiple incompressible Navier-Stokes equations (NSE) at one pass. The one-leg, two-step methods designed by Dahlquist, Liniger, and Nevanlinna…
Stochastic Gradient Langevin Dynamics (SGLD) is a sampling scheme for Bayesian modeling adapted to large datasets and models. SGLD relies on the injection of Gaussian Noise at each step of a Stochastic Gradient Descent (SGD) update. In this…
A version of the Dynamical Systems Gradient Method for solving ill-posed nonlinear monotone operator equations is studied in this paper. A discrepancy principle is proposed and justified. A numerical experiment was carried out with the new…
We consider the stochastic gradient descent (SGD) algorithm driven by a general stochastic sequence, including i.i.d noise and random walk on an arbitrary graph, among others; and analyze it in the asymptotic sense. Specifically, we employ…
It is well known that second order homogeneous linear ordinary differential equations with slowly varying coefficients admit slowly varying phase functions. This observation underlies the Liouville-Green method and many other techniques for…