PSelInv -- A Distributed Memory Parallel Algorithm for Selected Inversion : the Symmetric Case

We describe an efficient parallel implementation of the selected inversion algorithm for distributed memory computer systems, which we call \texttt{PSelInv}. The \texttt{PSelInv} method computes selected elements of a general sparse matrix $A$ that can be decomposed as $A = LU$, where $L$ is lower triangular and $U$ is upper triangular. The implementation described in this paper focuses on the case of sparse symmetric matrices. It contains an interface that is compatible with the distributed memory parallel sparse direct factorization \texttt{SuperLU\_DIST}. However, the underlying data structure and design of \texttt{PSelInv} allows it to be easily combined with other factorization routines such as \texttt{PARDISO}. We discuss general parallelization strategies such as data and task distribution schemes. In particular, we describe how to exploit the concurrency exposed by the elimination tree associated with the $LU$ factorization of $A$. We demonstrate the efficiency and accuracy of \texttt{PSelInv} by presenting a number of numerical experiments. In particular, we show that \texttt{PSelInv} can run efficiently on more than $4,000$ cores for a modestly sized matrix. We also demonstrate how \texttt{PSelInv} can be used to accelerate large-scale electronic structure calculations.