Related papers: Step Sizes for Strong Stability Preservation with …
Single-call stochastic extragradient methods, like stochastic past extragradient (SPEG) and stochastic optimistic gradient (SOG), have gained a lot of interest in recent years and are one of the most efficient algorithms for solving…
A theoretical, and potentially also practical, problem with stochastic gradient descent is that trajectories may escape to infinity. In this note, we investigate uniform boundedness properties of iterates and function values along the…
Differentially private stochastic gradient descent (DP-SGD) has become the standard algorithm for training machine learning models with rigorous privacy guarantees. Despite its widespread use, the theoretical understanding of its long-run…
We consider a Lattice Boltzmann type discrete velocity model in the low Mach number scaling and develop a corresponding numerical scheme that remains uniformly valid across all regimes of the mean free path, from the kinetic to the…
Summation-by-parts (SBP) finite difference methods have several desirable properties for second-order wave equations. They combine the computational efficiency of narrow-stencil finite difference operators with provable stability on…
This paper proposes a thorough theoretical analysis of Stochastic Gradient Descent (SGD) with non-increasing step sizes. First, we show that the recursion defining SGD can be provably approximated by solutions of a time inhomogeneous…
We introduce a family of stochastic optimization methods based on the Runge-Kutta-Chebyshev (RKC) schemes. The RKC methods are explicit methods originally designed for solving stiff ordinary differential equations by ensuring that their…
We consider quadrature formulas of high order in time based on Radau-type, L-stable implicit Runge-Kutta schemes to solve time dependent stiff PDEs. Instead of solving a large nonlinear system of equations, we develop a method that performs…
We demonstrate that the clustering statistics and the corresponding phase transition to non-equilibrium clustering found in many experiments and simulation studies with self-propelled particles (SPPs) with alignment can be obtained from a…
Dynamical systems with sub-processes evolving on many different time scales are ubiquitous in applications. Their efficient solution is greatly enhanced by automatic time step variation. This paper is concerned with the theory, construction…
We study a fixed step-size noisy distributed gradient descent algorithm for solving optimization problems in which the objective is a finite sum of smooth but possibly non-convex functions. Random perturbations are introduced to the…
In this note we propose and analyze novel implicit-explicit methods based on second order strong stability preserving multistep time discretizations. Several schemes are developed, and a linear stability analysis is performed to study their…
This paper aims to investigate the numerical approximation of a general second order parabolic stochastic partial differential equation(SPDE) driven by multiplicative and additive noise. Our main interest is on such SPDEs where the…
The stable principal component pursuit (SPCP) problem is a non-smooth convex optimization problem, the solution of which has been shown both in theory and in practice to enable one to recover the low rank and sparse components of a matrix…
We consider a generic convex-concave saddle point problem with separable structure, a form that covers a wide-ranged machine learning applications. Under this problem structure, we follow the framework of primal-dual updates for saddle…
We investigate the problem of finding second-order stationary points (SOSP) in differentially private (DP) stochastic non-convex optimization. Existing methods suffer from two key limitations: (i) inaccurate convergence error rate due to…
We study the performance of stochastic first-order methods for finding saddle points of convex-concave functions. A notorious challenge faced by such methods is that the gradients can grow arbitrarily large during optimization, which may…
In this work, we conduct a systematic study of stochastic saddle point problems (SSP) and stochastic variational inequalities (SVI) under the constraint of $(\epsilon,\delta)$-differential privacy (DP) in both Euclidean and non-Euclidean…
This paper investigates numerical methods for solving stochastic linear quadratic (SLQ) optimal control problems governed by stochastic partial differential equations (SPDEs). Two distinct approaches, the open-loop and closed-loop ones, are…
This paper presents stability and accuracy analysis of a high-order explicit time stepping scheme introduced by \cite[Section 2.2]{Buvoli2019}, which exhibits superior stability compared to classical Adams-Bashforth. A conjecture that is…