Related papers: Direct Runge-Kutta Discretization Achieves Acceler…
We suggest a novel approach for the efficient and reliable approximation of the Pareto front of sufficiently smooth unconstrained bi-criteria optimization problems. Optimality conditions formulated for weighted sum scalarizations of the…
In this paper we consider time-dependent PDEs discretized by a special class of Physics Informed Neural Networks whose design is based on the framework of Runge--Kutta and related time-Galerkin discretizations. The primary motivation for…
We develop a distributed algorithm for convex Empirical Risk Minimization, the problem of minimizing large but finite sum of convex functions over networks. The proposed algorithm is derived from directly discretizing the second-order…
We provide a novel accelerated first-order method that achieves the asymptotically optimal convergence rate for smooth functions in the first-order oracle model. To this day, Nesterov's Accelerated Gradient Descent (AGD) and variations…
We propose a novel second-order ODE as the continuous-time limit of a Riemannian accelerated gradient-based method on a manifold with curvature bounded from below. This ODE can be seen as a generalization of the ODE derived for Euclidean…
In this paper, we propose Galerkin-Legendre spectral method with implicit Runge-Kutta method for solving the unsteady two-dimensional Schrodinger equation with nonhomogeneous Dirichlet boundary conditions and initial condition. We apply a…
Convergence analysis of accelerated first-order methods for convex optimization problems are presented from the point of view of ordinary differential equation solvers. A new dynamical system, called Nesterov accelerated gradient flow, has…
In this paper we study the stability of explicit finite difference discretizations of linear advection-diffusion equations (ADE) with arbitrary order of accuracy in the context of method of lines. The analysis first focuses on the stability…
Gradient-based optimization algorithms can be studied from the perspective of limiting ordinary differential equations (ODEs). Motivated by the fact that existing ODEs do not distinguish between two fundamentally different…
We introduce two block coordinate descent algorithms for solving optimization problems with ordinary differential equations (ODEs) as dynamical constraints. The algorithms do not need to implement direct or adjoint sensitivity analysis…
In this paper, explicit stable integrators based on symplectic and contact geometries are proposed for a non-autonomous ordinarily differential equation (ODE) found in improving convergence rate of Nesterov's accelerated gradient method.…
We study stochastic optimization from a joint continuous-discrete point of view. Starting from a second-order stochastic differential equation interpreted as a noisy accelerated gradient flow, we discretize the dynamics by a fully implicit…
In this paper we discuss the use of implicit Runge-Kutta schemes for the time discretization of optimal control problems with evolution equations. The specialty of the considered discretizations is that the discretizations schemes for the…
We study Runge-Kutta methods for rough differential equations which can be used to calculate solutions to stochastic differential equations driven by processes that are rougher than a Brownian motion. We use a Taylor series representation…
Some continuous optimization methods can be connected to ordinary differential equations (ODEs) by taking continuous limits, and their convergence rates can be explained by the ODEs. However, since such ODEs can achieve any convergence rate…
The minimization of the loss function is of paramount importance in deep neural networks. On the other hand, many popular optimization algorithms have been shown to correspond to some evolution equation of gradient flow type. Inspired by…
Viewing optimization methods as numerical integrators for ordinary differential equations (ODEs) provides a thought-provoking modern framework for studying accelerated first-order optimizers. In this literature, acceleration is often…
We deal with optimal approximation of solutions of ODEs under local Lipschitz condition and inexact discrete information about the right-hand side functions. We show that the randomized two-stage Runge-Kutta scheme is the optimal method…
We consider unconstrained minimization of smooth convex functions. We propose a novel variational perspective using forced Euler-Lagrange equation that allows for studying high-resolution ODEs. Through this, we obtain a faster convergence…
Many time-dependent partial differential equations (PDEs) can be transformed into an ordinary differential equations (ODEs) containing moderately stiff and non-stiff terms after spatial semi-discretization. In the present paper, we…