Related papers: Step Sizes for Strong Stability Preservation with …
In this paper, we consider a new approach for semi-discretization in time and spatial discretization of a class of semi-linear stochastic partial differential equations (SPDEs) with multiplicative noise. The drift term of the SPDEs is only…
Stochastic bilevel optimization (SBO) has been integrated into many machine learning paradigms recently, including hyperparameter optimization, meta learning, and reinforcement learning. Along with the wide range of applications, there have…
This article considers estimation of constant and time-varying coefficients in nonlinear ordinary differential equation (ODE) models where analytic closed-form solutions are not available. The numerical solution-based nonlinear least…
High-dimensional Kronecker-structured estimation faces a conflict between non-convex scaling ambiguities and statistical robustness. The arbitrary factor scaling distorts gradient magnitudes, rendering standard fixed-threshold robust…
This paper aims to investigate the numerical approximation of semilinear non-autonomous stochastic partial differential equations (SPDEs) driven by multiplicative or additive noise. Such equations are more realistic than autonomous SPDEs…
This paper focuses on the strong convergence rate of both Runge--Kutta methods and simplified step-$N$ Euler schemes for stochastic differential equations driven by multi-dimensional fractional Brownian motions with $H\in(\frac12,1)$. Based…
This paper presents the Safe Sequential Quadratically Constrained Quadratic Programming (SS-QCQP) algorithm, a first-order method for smooth inequality-constrained nonconvex optimization that guarantees feasibility at every iteration. The…
For the approximation of solutions for stochastic partial differential equations, numerical methods that obtain a high order of convergence and at the same time involve reasonable computational cost are of particular interest. We therefore…
The parametric instability arising when ordinary differential equations (ODEs) are numerically integrated with Runge-Kutta-Nystr\"om (RKN) methods with varying step sizes is investigated. It is shown that when linear constant coefficient…
A new explicit stabilized scheme of weak order one for stiff and ergodic stochastic differential equations (SDEs) is introduced. In the absence of noise, the new method coincides with the classical deterministic stabilized scheme (or…
In view of a direct and simple improvement of vanilla SGD, this paper presents a fine-tuning of its step-sizes in the mini-batch case. For doing so, one estimates curvature, based on a local quadratic model and using only noisy gradient…
We present a comprehensive theoretical analysis of first-order methods for escaping strict saddle points in smooth non-convex optimization. Our main contribution is a Perturbed Saddle-escape Descent (PSD) algorithm with fully explicit…
This paper presents the first sufficient conditions that guarantee the stability and almost sure convergence of multi-timescale stochastic approximation (SA) iterates. It extends the existing results on one-timescale and two-timescale SA…
The analysis on the global stability of Riemannian gradient descent method in manifold optimization (i.e., it avoids strict saddle points for almost all initializations) due to Lee et al. (Math. Program. 176:311-337) is corrected. Moreover,…
Many popular learning-rate schedules for deep neural networks combine a decaying trend with local perturbations that attempt to escape saddle points and bad local minima. We derive convergence guarantees for bandwidth-based step-sizes, a…
Machine learned partial differential equation (PDE) solvers trade the reliability of standard numerical methods for potential gains in accuracy and/or speed. The only way for a solver to guarantee that it outputs the exact solution is to…
Modern data science applications demand solving large-scale optimization problems. The prevalent approaches are first-order methods, valued for their scalability. These methods are implemented to tackle highly irregular problems where…
Maximum bound principle (MBP) is an important property for a large class of semilinear parabolic equations, in the sense that the time-dependent solution of the equation with appropriate initial and boundary conditions and nonlinear…
This paper introduces a coordinate descent version of the V\~u-Condat algorithm. By coordinate descent, we mean that only a subset of the coordinates of the primal and dual iterates is updated at each iteration, the other coordinates being…
We present a family of multistep integrators based on the Adams-Bashforth methods. These schemes can be constructed for arbitrary convergence order with arbitrary step size variation. The step size can differ between different subdomains of…