The AZ algorithm for least squares systems with a known incomplete generalized inverse

We introduce an algorithm for the least squares solution of a rectangular linear system $Ax=b$, in which $A$ may be arbitrarily ill-conditioned. We assume that a complementary matrix $Z$ is known such that $A - AZ^*A$ is numerically low rank. Loosely speaking, $Z^*$ acts like a generalized inverse of $A$ up to a numerically low rank error. We give several examples of $(A,Z)$ combinations in function approximation, where we can achieve high-order approximations in a number of non-standard settings: the approximation of functions on domains with irregular shapes, weighted least squares problems with highly skewed weights, and the spectral approximation of functions with localized singularities. The algorithm is most efficient when $A$ and $Z^*$ have fast matrix-vector multiplication and when the numerical rank of $A - AZ^*A$ is small.