Related papers: Strong stability preserving explicit linear multis…

This paper is concerned with the theory, construction and application of implicit Peer two-step methods that are super-convergent for variable stepsizes, i.e., preserve their classical order achieved for uniform stepsizes when applied to…

Stabilized methods (also called Chebyshev methods) are explicit methods with extended stability domains along the negative real axis. These methods are intended for large mildly stiff problems, originating mainly from parabolic PDEs. In…

Strong stability preserving (SSP) Runge-Kutta methods are desirable when evolving in time problems that have discontinuities or sharp gradients and require nonlinear non-inner-product stability properties to be satisfied. Unlike the case…

The Barzilai-Borwein (BB) step sizes have a profound impact on gradient descent methods. In this work, we propose two new gradient step sizes: one longer than the original long BB step size, and the other shorter than the original short BB…

We study robust convex quadratic programs where the uncertain problem parameters can contain both continuous and integer components. Under the natural boundedness assumption on the uncertainty set, we show that the generic problems are…

This paper focuses on the question of how unconditional stability can be achieved via multistep ImEx schemes, in practice problems where both the implicit and explicit terms are allowed to be stiff. For a class of new ImEx multistep schemes…

Singularly perturbed systems (SPSs) are prevalent in engineering applications, where numerically solving their initial value problems (IVPs) is challenging due to stiffness arising from multiple time scales. Classical explicit methods…

This paper studies multistep methods for the integration of reversible dynamical systems, with particular emphasis on the planar Kepler problem. It has previously been shown by Cano & Sanz-Serna that reversible linear multisteps for…

The popularity of bi-level optimization (BO) in deep learning has spurred a growing interest in studying gradient-based BO algorithms. However, existing algorithms involve two coupled learning rates that can be affected by approximation…

Stochastic Gradient Descent (SGD) is a popular tool in training large-scale machine learning models. Its performance, however, is highly variable, depending crucially on the choice of the step sizes. Accordingly, a variety of strategies for…

We study step-wise time approximations of non-linear hyperbolic initial value problems. The technique used here is a generalization of the minimizing movements method, using two time-scales: one for velocity, the other (potentially much…

We propose a stochastic variant of the classical Polyak step-size (Polyak, 1987) commonly used in the subgradient method. Although computing the Polyak step-size requires knowledge of the optimal function values, this information is readily…

The dynamical stability of optimization methods at the vicinity of minima of the loss has recently attracted significant attention. For gradient descent (GD), stable convergence is possible only to minima that are sufficiently flat w.r.t.…

We study the stability of explicit one-step integration schemes for the linear finite element approximation of linear parabolic equations. The derived bound on the largest permissible time step is tight for any mesh and any diffusion matrix…

We introduce a new $\mathbf F$-modulated energy stability framework for general linear multistep methods. We showcase the theory for the two dimensional molecular beam epitaxy model with no slope selection which is a prototypical gradient…

We present a version of the stochastic maximum principle (SMP) for ergodic control problems. In particular we give necessary (and sufficient) conditions for optimality for controlled dissipative systems in finite dimensions. The strategy we…

This paper presents a comprehensive analysis of a broad range of variations of the stochastic proximal point method (SPPM). Proximal point methods have attracted considerable interest owing to their numerical stability and robustness…

A scheme for stabilizing stochastic approximation iterates by adaptively scaling the step sizes is proposed and analyzed. This scheme leads to the same limiting differential equation as the original scheme and therefore has the same…

In this paper, we focus on the problem of existence and computing of small and large stable models. We show that for every fixed integer k, there is a linear-time algorithm to decide the problem LSM (large stable models problem): does a…

Many popular learning-rate schedules for deep neural networks combine a decaying trend with local perturbations that attempt to escape saddle points and bad local minima. We derive convergence guarantees for bandwidth-based step-sizes, a…