Related papers: SRKCD: a stabilized Runge-Kutta method for stochas…
We propose a stochastic gradient framework for solving stochastic composite convex optimization problems with (possibly) infinite number of linear inclusion constraints that need to be satisfied almost surely. We use smoothing and homotopy…
This paper investigates the energy conservation properties of explicit Runge--Kutta (RK) time discretizations for autonomous skew-symmetric systems. For linear problems, we present a general framework for constructing RK methods in which…
Coordinate descent algorithms are widely used in machine learning and large-scale data analysis due to their strong optimality guarantees and impressive empirical performance in solving non-convex problems. In this work, we introduce Block…
This paper considers the decision-dependent optimization problem, where the data distributions react in response to decisions affecting both the objective function and linear constraints. We propose a new method termed repeated projected…
Convenient, easy to implement stochastic integration methods are developed on the basis of abstract one-step deterministic order $p$ integration techniques. The abstraction as an arbitrary one step map allows the inspection of easy to…
In this paper, we propose a new stochastic column-block gradient descent method for solving nonlinear systems of equations. It has a descent direction and holds an approximately optimal step size obtained through an optimization problem. We…
Nonconvex optimization is central in solving many machine learning problems, in which block-wise structure is commonly encountered. In this work, we propose cyclic block coordinate methods for nonconvex optimization problems with…
We present unconditionally energy stable Runge-Kutta (RK) discontinuous Galerkin (DG) schemes for solving a class of fourth order gradient flows. Our algorithm is geared toward arbitrarily high order approximations in both space and time,…
In this technical note a general procedure is described to construct internally consistent splitting methods for the numerical solution of differential equations, starting from matching pairs of explicit and diagonally implicit Runge-Kutta…
This paper presents an algorithmic framework for solving unconstrained stochastic optimization problems using only stochastic function evaluations. We employ central finite-difference based gradient estimation methods to approximate the…
In this paper, we propose a novel sufficient decrease technique for stochastic variance reduced gradient descent methods such as SVRG and SAGA. In order to make sufficient decrease for stochastic optimization, we design a new sufficient…
A stochastic-gradient-based interior-point algorithm for minimizing a continuously differentiable objective function (that may be nonconvex) subject to bound constraints is presented, analyzed, and demonstrated through experimental results.…
Recently, a new class of second order Runge-Kutta methods for It\^o stochastic differential equations with a multidimensional Wiener process was introduced by R\"o{\ss}ler. In contrast to second order methods earlier proposed by other…
Runge-Kutta methods are a popular class of numerical methods for solving ordinary differential equations. Every Runge-Kutta method is characterized by two basic parameters: its order, which measures the accuracy of the solution it produces,…
High order spatial discretizations with monotonicity properties are often desirable for the solution of hyperbolic PDEs. These methods can advantageously be coupled with high order strong stability preserving time discretizations. The…
This paper introduces a new proximal stochastic gradient method with variance reduction and stabilization for minimizing the sum of a convex stochastic function and a group sparsity-inducing regularization function. Since the method may be…
The minimization of the loss function is of paramount importance in deep neural networks. On the other hand, many popular optimization algorithms have been shown to correspond to some evolution equation of gradient flow type. Inspired by…
We present second-order optimally stable Implicit-Explicit (IMEX) Runge-Kutta (RK) schemes with application to a modified set of shallow water equations that can be used to model the dynamics of lava flows. The schemes are optimally stable…
In this paper, we study the sequential convex programming method with monotone line search (SCP$_{ls}$) in [46] for a class of difference-of-convex (DC) optimization problems with multiple smooth inequality constraints. The SCP$_{ls}$ is a…
Stochastic Optimization is a cornerstone of operations research, providing a framework to solve optimization problems under uncertainty. Despite the development of numerous algorithms to tackle these problems, several persistent challenges…