Related papers: Two-derivative error inhibiting schemes with post-…

For many systems of differential equations modeling problems in science and engineering, there are natural splittings of the right hand side into two parts, one non-stiff or mildly stiff, and the other one stiff. For such systems…

Local convergence analysis of the augmented Lagrangian method (ALM) is established for a large class of composite optimization problems with nonunique Lagrange multipliers under a second-order sufficient condition. We present a new…

An algorithm is proposed for solving optimization problems arising in neural network training for supervised learning. The unique feature of the algorithm is the use of an auxiliary loss, in addition to the original loss employed for model…

Stochastic gradient-based descent (SGD), have long been central to training large language models (LLMs). However, their effectiveness is increasingly being questioned, particularly in large-scale applications where empirical evidence…

Interior point methods solve small to medium sized problems to high accuracy in a reasonable amount of time. However, for larger problems as well as stochastic problems, one needs to use first-order methods such as stochastic gradient…

This paper proposes the method to optimize restriction and prolongation operators in the two-grid method. The proposed method is straightforwardly extended to the geometric multigrid method (GMM). GMM is used in solving discretized partial…

The alternating direction method of multipliers (ADMM) is widely used for solving large-scale semidefinite programs (SDPs), yet on instances with multiple primal-dual optimal solution pairs, it often enters prolonged slow-convergence…

We study the high-order local discontinuous Galerkin (LDG) method for the $p$-Laplace equation. We reformulate our spatial discretization as an equivalent convex minimization problem and use a preconditioned gradient descent method as the…

To facilitate the numerical analysis of particle methods, we derive truncation error estimates for the approximate operators in a generalized particle method. Here, a generalized particle method is defined as a meshfree numerical method…

This paper addresses problems of second-order cone programming important in optimization theory and applications. The main attention is paid to the augmented Lagrangian method (ALM) for such problems considered in both exact and inexact…

Among the methods for solving ODE-IVPs, the class of General Linear Methods (GLMs) is able to encompass most of them, ranging from Linear Multistep Formulae (LMF) to RK formulae. Moreover, it is possible to obtain methods able to overcome…

We introduce and analyze a post-processing for a family of variational space-time approximations to wave problems. The discretization in space and time is based on continuous finite element methods. The post-processing lifts the fully…

First-order methods such as stochastic gradient descent (SGD) are currently the standard algorithm for training deep neural networks. Second-order methods, despite their better convergence rate, are rarely used in practice due to the…

We consider high-order splitting schemes for large-scale differential Riccati equations. Such equations arise in many different areas and are especially important within the field of optimal control. In the large-scale case, it is critical…

Most modern supervised statistical/machine learning (ML) methods are explicitly designed to solve prediction problems very well. Achieving this goal does not imply that these methods automatically deliver good estimators of causal…

Two residual-type error estimators for the mortar staggered discontinuous Galerkin discretizations of second order elliptic equations are developed. Both error estimators are proved to be reliable and efficient. Key to the derivation of the…

This paper introduces two decomposition-based methods for two-block mixed-integer linear programs (MILPs), which aim to take advantage of separable structures of the original problem by solving a sequence of lower-dimensional MILPs. The…

In this paper, we develop a low-rank method with high-order temporal accuracy using spectral deferred correction (SDC) to compute linear matrix differential equations. In [1], a low rank numerical method is proposed to correct the modeling…

In this paper, we consider a general stochastic optimization problem which is often at the core of supervised learning, such as deep learning and linear classification. We consider a standard stochastic gradient descent (SGD) method with a…

In this paper, we develop a class of high order conservative semi-Lagrangian (SL) discontinuous Galerkin (DG) methods for solving multi-dimensional linear transport equations. The methods rely on a characteristic Galerkin weak formulation,…