An exact $\sin\Theta$ formula for matrix perturbation analysis and its applications

In this paper, we establish a useful set of formulae for the $\sin\Theta$ distance between the original and the perturbed singular subspaces. These formulae explicitly show that how the perturbation of the original matrix propagates into singular vectors and singular subspaces, thus providing a direct way of analyzing them. Following this, we derive a collection of new results on SVD perturbation related problems, including a tighter bound on the $\ell_{2,\infty}$ norm of the singular vector perturbation errors under Gaussian noise, a new stability analysis of the Principal Component Analysis and an error bound on the singular value thresholding operator. For the latter two, we consider the most general rectangular matrices with full matrix rank.