A Convergence Study for Reduced Rank Extrapolation on Nonlinear Systems

Reduced Rank Extrapolation (RRE) is a polynomial type method used to accelerate the convergence of sequences of vectors $\{\boldsymbol{x}_m\}$. It is applied successfully in different disciplines of science and engineering in the solution of large and sparse systems of linear and nonlinear equations of very large dimension. If $\boldsymbol{s}$ is the solution to the system of equations $\boldsymbol{x}=\boldsymbol{f}(\boldsymbol{x})$, first, a vector sequence $\{\boldsymbol{x}_m\}$ is generated via the fixed-point iterative scheme $\boldsymbol{x}_{m+1}=\boldsymbol{f}(\boldsymbol{x}_m)$, $m=0,1,\ldots,$ and next, RRE is applied to this sequence to accelerate its convergence. RRE produces approximations $\boldsymbol{s}_{n,k}$ to $\boldsymbol{s}$ that are of the form $\boldsymbol{s}_{n,k}=\sum^k_{i=0}\gamma_i\boldsymbol{x}_{n+i}$ for some scalars $\gamma_i$ depending (nonlinearly) on $\boldsymbol{x}_n, \boldsymbol{x}_{n+1},\ldots,\boldsymbol{x}_{n+k+1}$ and satisfying $\sum^k_{i=0}\gamma_i=1$. The convergence properties of RRE when applied in conjunction with linear $\boldsymbol{f}(\boldsymbol{x})$ have been analyzed in different publications. In this work, we discuss the convergence of the $\boldsymbol{s}_{n,k}$ obtained from RRE with nonlinear $\boldsymbol{f}(\boldsymbol{x})$ (i)\,when $n\to\infty$ with fixed $k$, and (ii)\,in two so-called {\em cycling} modes.