Related papers: SRKCD: a stabilized Runge-Kutta method for stochas…
Explicit Runge-Kutta schemes with large stable step sizes are developed for integration of high order spectral difference spatial discretization on quadrilateral grids. The new schemes permit an effective time step that is substantially…
Stochastic convex optimization is a basic and well studied primitive in machine learning. It is well known that convex and Lipschitz functions can be minimized efficiently using Stochastic Gradient Descent (SGD). The Normalized Gradient…
In this paper, we extend the Paired-Explicit Runge-Kutta schemes by Vermeire et. al. to fourth-order of consistency. Based on the order conditions for partitioned Runge-Kutta methods we motivate a specific form of the Butcher arrays which…
In this paper, we show that simple {Stochastic} subGradient Decent methods with multiple Restarting, named {\bf RSGD}, can achieve a \textit{linear convergence rate} for a class of non-smooth and non-strongly convex optimization problems…
We study the application of the generalized convolution quadrature (gCQ) based on Runge--Kutta methods to approximate the solution of an important class of sectorial problems. The gCQ generalizes Lubich's original convolution quadrature…
We further develop a simple modification of Runge--Kutta methods that guarantees conservation or stability with respect to any inner-product norm. The modified methods can be explicit and retain the accuracy and stability properties of the…
Consider convex optimization problems subject to a large number of constraints. We focus on stochastic problems in which the objective takes the form of expected values and the feasible set is the intersection of a large number of convex…
We introduce a class of high order accurate, semi-implicit Runge-Kutta schemes in the general setting of evolution equations that arise as gradient flow for a cost function, possibly with respect to an inner product that depends on the…
We consider stochastic convex optimization problems with affine constraints and develop several methods using either primal or dual approach to solve it. In the primal case, we use a special penalization technique to make the initial…
In this paper, we consider continuous-time stochastic optimal control problems where the cost is evaluated through a coherent risk measure. We provide an explicit gradient descent-ascent algorithm which applies to problems subject to…
This paper considers master equations for Markovian kinetic schemes that possess the detailed balance property. Chemical kinetics, as a prime example, often yields large-scale, highly stiff equations. Based on chemical intuitions, Sumiya et…
The study of optimal control problems under uncertainty plays an important role in scientific numerical simulations. This class of optimization problems is strongly utilized in engineering, biology and finance. In this paper, a stochastic…
This paper considers a class of constrained stochastic composite optimization problems whose objective function is given by the summation of a differentiable (possibly nonconvex) component, together with a certain non-differentiable (but…
Stochastic coordinate descent algorithms are efficient methods in which each iterate is obtained by fixing most coordinates at their values from the current iteration, and approximately minimizing the objective with respect to the remaining…
The recently-introduced relaxation approach for Runge-Kutta methods can be used to enforce conservation of energy in the integration of Hamiltonian systems. We study the behavior of implicit and explicit relaxation Runge-Kutta methods in…
A wide range of physical phenomena exhibit auxiliary admissibility criteria, such as conservation of entropy or various energies, which arise implicitly under the exact solution of their governing PDEs. However, standard temporal schemes,…
Stochastic-gradient-based optimization has been a core enabling methodology in applications to large-scale problems in machine learning and related areas. Despite the progress, the gap between theory and practice remains significant, with…
In this paper, we develop a higher order symmetric partitioned Runge-Kutta method for a coupled system of differential equations on Lie groups. We start with a discussion on partitioned Runge-Kutta methods on Lie groups of arbitrary order.…
In this paper, we propose first-order feasible methods for difference-of-convex (DC) programs with smooth inequality and simple geometric constraints. Our strategy for maintaining feasibility of the iterates is based on a "retraction" idea…
We study local complexity measures for stochastic convex optimization problems, providing a local minimax theory analogous to that of H\'{a}jek and Le Cam for classical statistical problems. We give complementary optimality results,…