Related papers: SRKCD: a stabilized Runge-Kutta method for stochas…
We consider randomized block coordinate stochastic mirror descent (RBSMD) methods for solving high-dimensional stochastic optimization problems with strongly convex objective functions. Our goal is to develop RBSMD schemes that achieve a…
Many relevant problems in the area of systems and control, such as controller synthesis, observer design and model reduction, can be viewed as optimization problems involving dynamical systems: for instance, maximizing performance in the…
Block coordinate descent methods and stochastic subgradient methods have been extensively studied in optimization and machine learning. By combining randomized block sampling with stochastic subgradient methods based on dual averaging, we…
In this paper we discuss the use of implicit Runge-Kutta schemes for the time discretization of optimal control problems with evolution equations. The specialty of the considered discretizations is that the discretizations schemes for the…
Nonconvex optimization underlies many modern machine learning and control tasks, where saddle points pose the dominant obstacle to reliable convergence in high-dimensional settings. Escaping these saddle points deterministically using…
In nonsmooth optimization, a negative subgradient is not necessarily a descent direction, making the design of convergent descent methods based on zeroth-order and first-order information a challenging task. The well-studied bundle methods…
The framework of inner product norm preserving relaxation Runge-Kutta methods (David I. Ketcheson, \emph{Relaxation Runge-Kutta Methods: Conservation and Stability for Inner-Product Norms}, SIAM Journal on Numerical Analysis, 2019) is…
Motivated by studies on fully discrete numerical schemes for linear hyperbolic conservation laws, we present a framework on analyzing the strong stability of explicit Runge-Kutta (RK) time discretizations for semi-negative autonomous linear…
High-order spatial discretizations with strong stability properties (such as monotonicity) are desirable for the solution of hyperbolic PDEs. Methods may be compared in terms of the strong stability preserving (SSP) time-step. We prove an…
The Runge--Kutta (RK) discontinuous Galerkin (DG) method is a mainstream numerical algorithm for solving hyperbolic equations. In this paper, we use the linear advection equation in one and two dimensions as a model problem to prove the…
Stochastic gradient descent is a canonical tool for addressing stochastic optimization problems, and forms the bedrock of modern machine learning and statistics. In this work, we seek to balance the fact that attenuating step-size is…
Stochastic gradient descent (SGD) is a simple and popular method to solve stochastic optimization problems which arise in machine learning. For strongly convex problems, its convergence rate was known to be O(\log(T)/T), by running SGD for…
In this paper we analyze the randomized block-coordinate descent (RBCD) methods proposed in [8,11] for minimizing the sum of a smooth convex function and a block-separable convex function. In particular, we extend Nesterov's technique…
We consider a distributionally robust formulation of stochastic optimization problems arising in statistical learning, where robustness is with respect to uncertainty in the underlying data distribution. Our formulation builds on…
A mixed accuracy framework for Runge--Kutta methods presented in Grant [JSC 2022] and applied to diagonally implicit Runge--Kutta (DIRK) methods can significantly speed up the computation by replacing the implicit solver by less expensive…
In this paper we present a novel randomized block coordinate descent method for the minimization of a convex composite objective function. The method uses (approximate) partial second-order (curvature) information, so that the algorithm…
Recent advances in convex optimization have leveraged computer-assisted proofs to develop optimized first-order methods that improve over classical algorithms. However, each optimized method is specially tailored for a particular problem…
Novel coordinate descent (CD) methods are proposed for minimizing nonconvex functions consisting of three terms: (i) a continuously differentiable term, (ii) a simple convex term, and (iii) a concave and continuous term. First, by extending…
A wide range of physical phenomena exhibit auxiliary admissibility criteria, such as conservation of entropy or various energies, which arise implicitly under exact solution of their governing PDEs. However, standard temporal schemes, such…
Stochastic compositional optimization generalizes classic (non-compositional) stochastic optimization to the minimization of compositions of functions. Each composition may introduce an additional expectation. The series of expectations may…