Related papers: The Communication-Hiding Conjugate Gradient Method…
Gaussian processes (GPs) are important probabilistic tools for inference and learning in spatio-temporal modelling problems such as those in climate science and epidemiology. However, existing GP approximations do not simultaneously support…
The nonlinear (preconditioned) conjugate gradient N(P)CG method and the locally optimal (preconditioned) minimal residual LO(P)MR method, both of which are used for the iterative computation of sparse approximate inverses (SPAIs) of…
The increasing size of input graphs for graph neural networks (GNNs) highlights the demand for using multi-GPU platforms. However, existing multi-GPU GNN systems optimize the computation and communication individually based on the…
This paper presents "Predictive Pipelined Decoding (PPD)," an approach that speeds up greedy decoding in Large Language Models (LLMs) while maintaining the exact same output as the original decoding. Unlike conventional strategies, PPD…
Standard Gaussian Process (GP) regression, a powerful machine learning tool, is computationally expensive when it is applied to large datasets, and potentially inaccurate when data points are sparsely distributed in a high-dimensional…
This paper surveys an important class of methods that combine iterative projection methods and variational regularization methods for large-scale inverse problems. Iterative methods such as Krylov subspace methods are invaluable in the…
Efficient simulation of nonlinear and dispersive free-surface flows governed by the incompressible Navier-Stokes equations remains a central challenge in ocean and coastal engineering. The computational bottleneck arises from solving a…
Massive amounts of data have led to the training of large-scale machine learning models on a single worker inefficient. Distributed machine learning methods such as Parallel-SGD have received significant interest as a solution to tackle…
Data-parallel SGD is the de facto algorithm for distributed optimization, especially for large scale machine learning. Despite its merits, communication bottleneck is one of its persistent issues. Most compression schemes to alleviate this…
Two new hybrid algorithms are proposed for large-scale linear discrete ill-posed problems in general-form regularization. They are both based on Krylov subspace inner-outer iterative algorithms. At each iteration, they need to solve a…
Least-squares Petrov--Galerkin (LSPG) model-reduction techniques such as the Gauss--Newton with Approximated Tensors (GNAT) method have shown promise, as they have generated stable, accurate solutions for large-scale turbulent, compressible…
Markov chain Monte Carlo (MCMC) methods are often used in clustering since they guarantee asymptotically exact expectations in the infinite-time limit. In finite time, though, slow mixing often leads to poor performance. Modern computing…
The mechanical properties of periodic microstructures are pivotal in various engineering applications. Homogenization theory is a powerful tool for predicting these properties by averaging the behavior of complex microstructures over a…
Stochastic gradient descent (SGD) is one of the most widely used optimization methods for solving various machine learning problems. SGD solves an optimization problem by iteratively sampling a few data points from the input data, computing…
We have developed an application and implemented parallel algorithms in order to provide a computational framework suitable for massively parallel supercomputers to study the unitary dynamics of quantum systems. We use renowned parallel…
We consider iterative methods for solving linear ill-posed problems with compact operator and right-hand side only available via noise-polluted measurements. Conjugate gradients (CG) applied to the normal equations with an appropriate…
In classical frameworks as the Euclidean space, positive definite kernels as well as their analytic properties are explicitly available and can be incorporated directly in kernel-based learning algorithms. This is different if the…
Stochastic Gradient Descent (SGD) is the key learning algorithm for many machine learning tasks. Because of its computational costs, there is a growing interest in accelerating SGD on HPC resources like GPU clusters. However, the…
We present a new short-recurrence reaidual-optimal Krylov subspace recycling method for sequences of Hermitian systems of linear equations with a fixed system matrix and changing right-hand sides. Such sequences of linear systems occur…
In distributed and federated learning algorithms, communication overhead is often reduced by performing multiple local updates between communication rounds. However, due to data heterogeneity across nodes and the local gradient noise within…