Related papers: Error Inhibiting Block One-Step Schemes for Ordina…

Four types of global error for initial value problems are considered in a common framework. They include classical forward error analysis and shadowing error analysis together with extensions of both to rescaling of time. To determine the…

Under interpolation-type assumptions such as the strong growth condition, stochastic optimization methods can attain convergence rates comparable to full-batch methods, but their performance, particularly for SGD, remains highly sensitive…

The primary focus of this paper is on designing an inexact first-order algorithm for solving constrained nonlinear optimization problems. By controlling the inexactness of the subproblem solution, we can significantly reduce the…

Convergence of classical parallel iterations is detected by performing a reduction operation at each iteration in order to compute a residual error relative to a potential solution vector. To efficiently run asynchronous iterations,…

Consistency models have recently been introduced to accelerate sampling from diffusion models by directly predicting the solution (i.e., data) of the probability flow ODE (PF ODE) from initial noise. However, the training of consistency…

Recently, a new class of BDF schemes proposed in [F. Huang and J. Shen, SIAM J Numer. Anal., 62.4, 1609--1637] for the parabolic type equations are studied in this paper. The basic idea is based on the Taylor expansions at time…

In simulations of fluid motion time accuracy has proven to be elusive. We seek highly accurate methods with strong enough stability properties to deal with the richness of scales of many flows. These methods must also be easy to implement…

Stochastic optimization methods have been hugely successful in making large-scale optimization problems feasible when computing the full gradient is computationally prohibitive. Using the theory of modified equations for numerical…

We propose new linear combinations of compositions of a basic second-order scheme with appropriately chosen coefficients to construct higher order numerical integrators for differential equations. They can be considered as a generalization…

Integration of Ordinary Differential Equations (ODEs) using Backward Difference formula (BDF) methods with p backward steps achieves order p accuracy if specific conditions are met. This work extends the composition technique with complex…

In this paper, we propose a class of super-schemes for efficiently solving nonlinear unconstrained optimization problems. The proposed approach introduces two novel choices of step-size parameters, leading to efficient descent directions…

We introduce a class of first-order methods for smooth constrained optimization that are based on an analogy to non-smooth dynamical systems. Two distinctive features of our approach are that (i) projections or optimizations over the entire…

In this article, we present a simple technique for boosting the order of accuracy of finite difference schemes for time dependent partial differential equations by optimally selecting the time step used to advance the numerical solution and…

This work proposes a framework, embedded within the Performance Estimation framework (PEP), for obtaining worst-case performance guarantees on stochastic first-order methods. Given a first-order method, a function class, and a noise model…

Stiff systems of ordinary differential equations (ODEs) arise in a wide range of scientific and engineering disciplines and are traditionally solved using implicit integration methods due to their stability and efficiency. However, these…

We propose new primal-dual decomposition algorithms for solving systems of inclusions involving sums of linearly composed maximally monotone operators. The principal innovation in these algorithms is that they are block-iterative in the…

In this paper, we introduce a new iterative method which we call one step back approach: the main idea is to anticipate the consequence of the iterative computation per coordinate and to optimize on the choice of the sequence of the…

First-order methods for minimization and saddle point (min-max) problems are widely used for solving large-scale problems, in particular arising in machine learning. The majority of works obtain favorable complexity guarantees of such…

In this paper, we study a stochastic strongly convex optimization problem and propose three classes of variable sample-size stochastic first-order methods including the standard stochastic gradient descent method, its accelerated variant,…

In this paper, we propose a generic and simple strategy for utilizing stochastic gradient information in optimization. The technique essentially contains two consecutive steps in each iteration: 1) computing and normalizing each block…