Related papers: Numerical Stability of s-step Enlarged Krylov Subs…
Iterative procedures for parameter estimation based on stochastic gradient descent allow the estimation to scale to massive data sets. However, in both theory and practice, they suffer from numerical instability. Moreover, they are…
High-quality reconstructions of signals and images with sharp edges are needed in a wide range of applications. To overcome the large dimensionality of the parameter space and the complexity of the regularization functional,…
There is widespread sentiment that it is not possible to effectively utilize fast gradient methods (e.g. Nesterov's acceleration, conjugate gradient, heavy ball) for the purposes of stochastic optimization due to their instability and error…
An efficient and robust linear scaling method is presented for large scale {\it ab initio} electronic structure calculations of a wide variety of materials including metals. The detailed short range and the effective long range…
We compare two approaches to compute a portion of the spectrum of dense symmetric definite generalized eigenproblems: one is based on the reduction to tridiagonal form, and the other on the Krylov-subspace iteration. Two large-scale…
Stabilized methods (also called Chebyshev methods) are explicit methods with extended stability domains along the negative real axis. These methods are intended for large mildly stiff problems, originating mainly from parabolic PDEs. In…
The implementation of a vast majority of machine learning (ML) algorithms boils down to solving a numerical optimization problem. In this context, Stochastic Gradient Descent (SGD) methods have long proven to provide good results, both in…
With the ever increasing computational power available and the development of high-performances computing, investigating the properties of realistic very large-scale nonlinear dynamical systems has been become reachable. It must be noted…
For the large-scale linear discrete ill-posed problem $\min\|Ax-b\|$ or $Ax=b$ with $b$ contaminated by Gaussian white noise, the Lanczos bidiagonalization based Krylov solver LSQR and its mathematically equivalent CGLS, the Conjugate…
Flexible Krylov methods are a common standpoint for inverse problems. In particular, they are used to address the challenges associated with explicit variational regularization when it goes beyond the two-norm, for example involving an…
The solution of sequences of shifted linear systems is a classic problem in numerical linear algebra, and a variety of efficient methods have been proposed over the years. Nevertheless, there still exist challenging scenarios witnessing a…
We review our recently developed methods for large-scale electronic structure calculations, both in one-electron theory and many-electron theory. The method are based on the density matrix representation, together with the Wannier state…
Analyzing massive spatial datasets using Gaussian process model poses computational challenges. This is a problem prevailing heavily in applications such as environmental modeling, ecology, forestry and environmental heath. We present a…
The use of block Krylov subspace methods for computing the solution to a sequence of shifted linear systems using subspace recycling was first proposed in [Soodhalter, SISC 2016], where a recycled shifted block GMRES algorithm (rsbGMRES)…
The resolvent Krylov subspace method builds approximations to operator functions $f(A)$ times a vector $v$. For the semigroup and related operator functions, this method is proved to possess the favorable property that the convergence is…
Stochastic gradient algorithms are often unstable when applied to functions that do not have Lipschitz-continuous and/or bounded gradients. Gradient clipping is a simple and effective technique to stabilize the training process for problems…
We introduce the Tensorized-and-Restricted Krylov (TReK) method, a simple and efficient algorithm for estimating covariance tensors with large observational sizes. TReK extends the conjugate gradient method to incorporate range…
Krylov subspace recycling is a powerful tool for solving long series of large, sparse linear systems that change slowly. In PDE constrained shape optimization, these appear naturally, as hundreds or more optimization steps are needed with…
Gradient compression is of growing interests for solving constrained optimization problems including compressed sensing, noisy recovery and matrix completion under limited communication resources and storage costs. Convergence analysis of…
Stochastic bilevel optimization (SBO) has been integrated into many machine learning paradigms recently, including hyperparameter optimization, meta learning, and reinforcement learning. Along with the wide range of applications, there have…