Related papers: Direct Runge-Kutta Discretization Achieves Acceler…
In this article, we propose novel boundary treatment algorithms to avoid order reduction when implicit-explicit Runge-Kutta time discretization is used for solving convection-diffusion-reaction problems with time-dependent Di\-richlet…
Neural Ordinary Differential Equations (Neural ODEs) represent continuous-time dynamics with neural networks, offering advancements for modeling and control tasks. However, training Neural ODEs requires solving differential equations at…
In this article, we derive fast and robust parallel-in-time preconditioned iterative methods for the all-at-once linear systems arising upon discretization of time-dependent PDEs. The discretization we employ is based on a Runge--Kutta…
A mixed accuracy framework for Runge--Kutta methods presented in [Grant, JSC 2022] has been shown to speed up the computation in diagonally implicit Runge--Kutta (DIRK) methods by using less expensive low accuracy approaches for the…
The problem of minimizing a separable convex function under linearly coupled constraints arises from various application domains such as economic systems, distributed control, and network flow. The main challenge for solving this problem is…
We show that the optimal complexity of Nesterov's smooth first-order optimization algorithm is preserved when the gradient is only computed up to a small, uniformly bounded error. In applications of this method to semidefinite programs,…
We present a new class of gradient-type optimization methods that extends vanilla gradient descent, mirror descent, Riemannian gradient descent, and natural gradient descent. Our approach involves constructing a surrogate for the objective…
In this paper, we study symmetric integrators for solving second-order ordinary differential equations on the basis of the notion of continuous-stage Runge-Kutta-Nystrom methods. The construction of such methods heavily relies on the…
This paper concerns an optimization algorithm for unconstrained non-convex problems where the objective function has sparse connections between the unknowns. The algorithm is based on applying a dissipation preserving numerical integrator,…
Models incorporating uncertain inputs, such as random forces or material parameters, have been of increasing interest in PDE-constrained optimization. In this paper, we focus on the efficient numerical minimization of a convex and smooth…
In this work we propose a differential geometric motivation for Nesterov's accelerated gradient method (AGM) for strongly-convex problems. By considering the optimization procedure as occurring on a Riemannian manifold with a natural…
The result after $N$ steps of an implicit Runge-Kutta time discretization of an inhomogeneous linear parabolic differential equation is computed, up to accuracy $\epsilon$, by solving only $$O\Big(\log N \log \frac1\epsilon \Big) $$ linear…
We introduce a second-order time discretization method for stiff kinetic equations. The method is asymptotic-preserving (AP) -- can capture the Euler limit without numerically resolving the small Knudsen number; and positivity-preserving --…
We study the problem of zero-order optimization of a strongly convex function. The goal is to find the minimizer of the function by a sequential exploration of its values, under measurement noise. We study the impact of higher order…
Binary optimization, a representative subclass of discrete optimization, plays an important role in mathematical optimization and has various applications in computer vision and machine learning. Usually, binary optimization problems are…
There has been significant interest in generalizations of the Nesterov accelerated gradient descent algorithm due to its improved performance guarantee compared to the standard gradient descent algorithm, and its applicability to large…
We propose a novel second-order optimization framework for training the emerging deep continuous-time models, specifically the Neural Ordinary Differential Equations (Neural ODEs). Since their training already involves expensive gradient…
We develop multi-step gradient methods for network-constrained optimization of strongly convex functions with Lipschitz-continuous gradients. Given the topology of the underlying network and bounds on the Hessian of the objective function,…
We interpret a wide range of flavors of Spectral Deferred Corrections (SDC) as Runge-Kutta methods (RKM). Using Butcher series, we show that the considered class of SDC methods achieve at least order p after p iterations compared to the…
We analyze the convergence rate of a family of inertial algorithms, which can be obtained by discretization of an inertial system with Hessian-driven damping. We recover a convergence rate, up to a factor of 2 speedup upon Nesterov's…