Related papers: Reducing Parallel Communication in Algebraic Multi…
Synchronous stochastic gradient descent (SGD) is the most common method used for distributed training of deep learning models. In this algorithm, each worker shares its local gradients with others and updates the parameters using the…
Algebraic multigrid (AMG) is conventionally applied in a black-box fashion, agnostic to the underlying geometry. In this work, we propose that using geometric information -- when available -- to assist with setting up the AMG hierarchy is…
We present an approach to constructing a practical coarsening algorithm and interpolation operator for the algebraic multigrid (AMG) method, tailored towards systems of partial differential equations (PDEs) with large near-kernels, such as…
In 1981 Hong and Kung proved a lower bound on the amount of communication needed to perform dense, matrix-multiplication using the conventional $O(n^3)$ algorithm, where the input matrices were too large to fit in the small, fast memory. In…
This paper introduces a novel approach to algebraic multigrid methods for large systems of linear equations coming from finite element discretizations of certain elliptic second order partial differential equations. Based on a discrete…
Modern large scale machine learning applications require stochastic optimization algorithms to be implemented on distributed computational architectures. A key bottleneck is the communication overhead for exchanging information such as…
Isogeometric analysis (IgA) offers enhanced approximation capabilities for the discretization of elliptic boundary-value problems, yet it results in large, sparse, and increasingly ill-conditioned linear systems due to higher…
Multigrid algorithms are among the fastest iterative methods known today for solving large linear and some non-linear systems of equations. Greatly optimized for serial operation, they still have a great potential for parallelism not fully…
Long-term state estimation over graphs remains challenging as current graph estimation methods scale poorly on large, long-term graphs. To address this, our work advances a current state-of-the-art graph sparsification algorithm, maximizing…
The partial reuse algebraic multigrid (AMG) setup cost amortization strategy is presented for the solution of non-steady state problems. The transfer operators are reused from the previous time steps, and the system matrices and the…
This paper describes a massively parallel algebraic multigrid method based on non-smoothed aggregation. It is especially suited for solving heterogeneous elliptic problems as it uses a greedy heuristic algorithm for the aggregation that…
Modern distributed training of machine learning models suffers from high communication overhead for synchronizing stochastic gradients and model parameters. In this paper, to reduce the communication complexity, we propose \emph{double…
Algebraic multigrid (AMG) solvers and preconditioners are some of the fastest numerical methods to solve linear systems, particularly in a parallel environment, scaling to hundreds of thousands of cores. Most AMG methods and theory assume a…
Algebraic Multigrid (AMG) is one of the most used iterative algorithms for solving large sparse linear equations $Ax=b$. In AMG, the coarse grid is a key component that affects the efficiency of the algorithm, the construction of which…
Distributed learning techniques such as federated learning have enabled multiple workers to train machine learning models together to reduce the overall training time. However, current distributed training algorithms (centralized or…
We address the communication overhead of distributed sparse matrix-(multiple)-vector multiplication in the context of large-scale eigensolvers, using filter diagonalization as an example. The basis of our study is a performance model which…
Strength-of-connection algorithms play a key role in algebraic multigrid (AMG). Specifically, they determine which matrix nonzeros are classified as weak and so ignored when coarsening matrix graphs and defining interpolation sparsity…
Data explosion and an increase in model size drive the remarkable advances in large-scale machine learning, but also make model training time-consuming and model storage difficult. To address the above issues in the distributed model…
The paper presents AMGCL -- an opensource C++ library implementing the algebraic multigrid method (AMG) for solution of large sparse linear systems of equations, usually arising from discretization of partial differential equations on an…
The current landscape of balanced graph partitioning is divided into high-quality but expensive multilevel algorithms and cheaper approaches with linear running time, such as single-level algorithms and streaming algorithms. We demonstrate…