Related papers: On the stability of optimization algorithms given …
We construct a continuous domain for temporal discretization of differential equations. By using this domain, and the domain of Lipschitz maps, we formulate a generalization of the Euler operator, which exhibits second-order convergence. We…
Ordinary differential equation (ODE) models of gradient-based optimization methods can provide insights into the dynamics of learning and inspire the design of new algorithms. Unfortunately, this thought-provoking perspective is weakened by…
We analyze the behavior of the Euler method for delay differential equations under nonstandard assumptions on the right-hand-side function f, when evaluations of f are corrupted by informational noise. We provide theoretical upper bounds on…
Second-order optimization methods exhibit fast convergence to critical points, however, in nonconvex optimization, these methods often require restrictive step-sizes to ensure a monotonically decreasing objective function. In the presence…
An implicit Euler--Maruyama method with non-uniform step-size applied to a class of stochastic partial differential equations is studied. A spectral method is used for the spatial discretization and the truncation of the Wiener process. A…
A quasi-second order scheme is developed to obtain approximate solutions of the shallow water equationswith bathymetry. The scheme is based on a staggered finite volume scheme for the space discretization:the scalar unknowns are located in…
We examine numerical rounding errors of some deterministic solvers for systems of ordinary differential equations (ODEs). We show that the accumulation of rounding errors results in a solution that is inherently random and we obtain the…
It is shown that for a parabolic problem with maximal $L^p$-regularity (for $1<p<\infty$), the time discretization by a linear multistep method or Runge--Kutta method has maximal $\ell^p$-regularity uniformly in the stepsize if the method…
We study the performance of stochastic first-order methods for finding saddle points of convex-concave functions. A notorious challenge faced by such methods is that the gradients can grow arbitrarily large during optimization, which may…
Classical neural ODEs trained with explicit methods are intrinsically limited by stability, crippling their efficiency and robustness for stiff learning problems that are common in graph learning and scientific machine learning. We present…
We consider the problem of designing uniformly stable first-order optimization algorithms for empirical risk minimization. Uniform stability is often used to obtain generalization error bounds for optimization algorithms, and we are…
Although Nesterov's accelerated gradient (NAG) methods have been studied from various perspectives, it remains unclear why the most popular forms of NAG must handle convex and strongly convex objective functions separately. Motivated by…
As a first step towards the numerical analysis of the stochastic primitive equations of the atmosphere and oceans, we study their time discretization by an implicit Euler scheme. From deterministic viewpoint the 3D Primitive Equations are…
Numerical schemes for the solution of the Euler equations have recently been developed, which involve the discretisation of the internal energy equation, with corrective terms to ensure the correct capture of shocks, and, more generally,…
We establish a discrete operator--theoretic framework for the analysis of implicit Euler and Lie--Trotter splitting schemes for delay differential equations (DDEs). Both schemes are formulated in terms of discrete resolvent operators acting…
A fully discrete Lagrangian scheme for solving a family of fourth order equations numerically is presented. The discretization is based on the equation's underlying gradient flow structure w.r.t. the $L^2$-Wasserstein distance, and adapts…
Differentially private zeroth-order optimization methods have recently gained popularity in private fine tuning of machine learning models due to their reduced memory requirements. Current approaches for privatizing zeroth-order methods…
In this paper we investigate the generalization error of gradient descent (GD) applied to an $\ell_2$-regularized OLS objective function in the linear model. Based on our analysis we develop new methodology for computationally tractable and…
A combination of implicit and explicit timestepping is analyzed for a system of ODEs motivated by ones arising from spatial discretizations of evolutionary partial differential equations. Loosely speaking, the method we consider is implicit…
Convergence analysis of accelerated first-order methods for convex optimization problems are presented from the point of view of ordinary differential equation solvers. A new dynamical system, called Nesterov accelerated gradient flow, has…