Related papers: A numerically stable communication-avoiding s-step…

In this paper we develop randomized Krylov subspace methods for efficiently computing regularized solutions to large-scale linear inverse problems. Building on the recently developed randomized Gram-Schmidt process, where sketched inner…

In this paper, we introduce a unified framework for nonlinear Krylov subspace methods (nlKrylov) to solve systems of nonlinear equations. Building on classical GCR-like/type linear Krylov solvers such as GMRESR, we generalize these…

Scalable QR factorization algorithms for solving least squares and eigenvalue problems are critical given the increasing parallelism within modern machines. We introduce a more general parallelization of the CholeskyQR2 algorithm and show…

While the filtered-x normalized least mean square (FxNLMS) algorithm is widely applied due to its simple structure and easy implementation for active noise control system, it faces two critical limitations: the fixed step-size causes a…

Adaptive cubic regularization methods for solving nonconvex problems need the efficient computation of the trial step, involving the minimization of a cubic model. We propose a new approach in which this model is minimized in a low…

The multiplication of two sparse matrices, known as SpGEMM, is a key kernel in scientific computing and large-scale data analytics, underpinning graph algorithms, machine learning, simulations, and computational biology, where sparsity is…

We present distributed algorithms for training dynamic Graph Neural Networks (GNN) on large scale graphs spanning multi-node, multi-GPU systems. To the best of our knowledge, this is the first scaling study on dynamic GNN. We devise…

Spiking Neural Networks (SNNs) are gaining significant traction in machine learning tasks where energy-efficiency is of utmost importance. Training such networks using the state-of-the-art back-propagation through time (BPTT) is, however,…

In recent years two Krylov subspace methods have been proposed for solving skew symmetric linear systems, one based on the minimum residual condition, the other on the Galerkin condition. We give new, algorithm-independent proofs that in…

We present a method for linear stability analysis of systems with parametric uncertainty formulated in the stochastic Galerkin framework. Specifically, we assume that for a model partial differential equation, the parameter is given in the…

One of the major bottlenecks for efficient deployment of neural network based recommendation systems is the memory footprint of their embedding tables. Although many neural network based recommendation systems could benefit from the faster…

Many challenging tasks in sensor networks, including sensor calibration, ranking of nodes, monitoring, event region detection, collaborative filtering, collaborative signal processing, {\em etc.}, can be formulated as a problem of solving a…

This paper studies multistep methods for the integration of reversible dynamical systems, with particular emphasis on the planar Kepler problem. It has previously been shown by Cano & Sanz-Serna that reversible linear multisteps for…

Many practical imaging systems suffer from uncertainty in acquisition geometry -- such as projection angles in computed tomography or sensor positions in photoacoustic tomography -- leading to nonlinear inverse problems that require joint…

We generalize the theory of underlying one-step methods to strictly stable general linear methods (GLMs) solving nonautonomous ordinary differential equations (ODEs) that satisfy a global Lipschitz condition. We combine this theory with the…

Gaussian processes are frequently deployed as part of larger machine learning and decision-making systems, for instance in geospatial modeling, Bayesian optimization, or in latent Gaussian models. Within a system, the Gaussian process model…

The Orthomin ( Omin ) and the Generalized Minimal Residual method ( GMRES ) are commonly used iterative methods for approximating the solution of non-symmetric linear systems. The s-step generalizations of these methods enhance their data…

Sparse matrix-matrix multiplication (SpGEMM) is a widely used kernel in various graph, scientific computing and machine learning algorithms. In this paper, we consider SpGEMMs performed on hundreds of thousands of processors generating…

For the large-scale linear discrete ill-posed problem $\min\|Ax-b\|$ or $Ax=b$ with $b$ contaminated by Gaussian white noise, the Lanczos bidiagonalization based Krylov solver LSQR and its mathematically equivalent CGLS, the Conjugate…

Decentralized nonconvex optimization has received increasing attention in recent years in machine learning due to its advantages in system robustness, data privacy, and implementation simplicity. However, three fundamental challenges in…