Related papers: Two-derivative error inhibiting schemes with post-…
In this work, we develop analysis and algorithms for a class of (stochastic) bilevel optimization problems whose lower-level (LL) problem is strongly convex and linearly constrained. Most existing approaches for solving such problems rely…
In this thesis, we propose new theoretical frameworks for the analysis of stochastic and distributed methods with error compensation and local updates. Using these frameworks, we develop more than 20 new optimization methods, including the…
This paper is concerned with developing and analyzing two novel implicit temporal discretization methods for the stochastic semilinear wave equations with multiplicative noise. The proposed methods are natural extensions of well-known…
We propose a new class of high-order time-marching schemes with dissipation user-control and unconditional stability for parabolic equations. High-order time integrators can deliver the optimal performance of highly-accurate and robust…
The use of the Preconditioned Conjugate Gradient (PCG) method for computing the Generalized Least Squares (GLS) estimator of the General Linear Model (GLM) is considered. The GLS estimator is expressed in terms of the solution of an…
In this paper, we propose a class of super-schemes for efficiently solving nonlinear unconstrained optimization problems. The proposed approach introduces two novel choices of step-size parameters, leading to efficient descent directions…
When balanced truncation is used for model order reduction, one has to solve a pair of Lyapunov equations for two Gramians and uses them to construct a reduced-order model. Although advances in solving such equations have been made, it is…
Model reduction is a powerful tool in dealing with numerical simulation of large scale dynamic systems for studying complex physical systems. Two major types of model reduction methods for linear time-invariant dynamic systems are Krylov…
An influential line of recent work has focused on the generalization properties of unregularized gradient-based learning procedures applied to separable linear classification with exponentially-tailed loss functions. The ability of such…
We consider in this work a system of two stochastic differential equations named the perturbed compositional gradient flow. By introducing a separation of fast and slow scales of the two equations, we show that the limit of the slow motion…
Logistic regression is one of the most popular methods in binary classification, wherein estimation of model parameters is carried out by solving the maximum likelihood (ML) optimization problem, and the ML estimator is defined to be the…
This work constructs and analyzes new efficient high-order two-derivative diagonally implicit Runge--Kutta (TDDIRK) schemes with optimized phase errors. Specifically, we present a convergence result for TDDIRK methods and investigate their…
In this work, we study and extend a class of semi-Lagrangian exponential methods, which combine exponential time integration techniques, suitable for integrating stiff linear terms, with a semi-Lagrangian treatment of nonlinear advection…
This paper presents a unified framework for supervised learning and inference procedures using the divide-and-conquer approach for high-dimensional correlated outcomes. We propose a general class of estimators that can be implemented in a…
Model order reduction involves constructing a reduced-order approximation of a high-order model while retaining its essential characteristics. This reduced-order model serves as a substitute for the original one in various applications such…
We consider a class of finite element approximations for fourth-order parabolic equations that can be written as a system of second-order equations by introducing an auxiliary variable. In our approach, we first solve a variational problem…
While first-order optimization methods such as stochastic gradient descent (SGD) are popular in machine learning (ML), they come with well-known deficiencies, including relatively-slow convergence, sensitivity to the settings of…
We consider a family of variational time discretizations that are generalizations of discontinuous Galerkin (dG) and continuous Galerkin-Petrov (cGP) methods. The family is characterized by two parameters. One describes the polynomial…
Lagrangian decomposition (LD) is a relaxation method that provides a dual bound for constrained optimization problems by decomposing them into more manageable sub-problems. This bound can be used in branch-and-bound algorithms to prune the…
The Alternating Direction Method of Multipliers (ADMM) has been studied for years. The traditional ADMM algorithm needs to compute, at each iteration, an (empirical) expected loss function on all training examples, resulting in a…