截断奇异值分解在一般高秩矩阵估计问题中的效力
机器学习
2017-11-07 v2 机器学习
数值分析
摘要
我们证明，给定一个在谱范数下接近一般高秩半正定矩阵 A 的估计 A ^ \widehat{A} A ∥ A ^ − A ∥ 2 ≤ δ \|\widehat{A}-A\|_2 \leq \delta ∥ A − A ∥ 2 ≤ δ A ^ \widehat{A} A Ω ( n max { ϵ − 4 , k 2 } μ 0 2 ∥ A ∥ F 2 log n σ k + 1 ( A ) 2 ) \Omega(\frac{n\max\{\epsilon^{-4},k^2\}\mu_0^2\|A\|_F^2\log n}{\sigma_{k+1}(A)^2}) Ω ( σ k + 1 ( A ) 2 n m a x { ϵ − 4 , k 2 } μ 0 2 ∥ A ∥ F 2 l o g n ) σ k + 1 ( A ) \sigma_{k+1}\left(A\right) σ k + 1 ( A ) μ 0 \mu_0 μ 0 ∥ A ^ k − A ∥ F ≤ ( 1 + O ( ϵ ) ) ∥ A − A k ∥ F \|\widehat{A}_k-A\|_F \leq (1+O(\epsilon))\|A-A_k\|_F ∥ A k − A ∥ F ≤ ( 1 + O ( ϵ )) ∥ A − A k ∥ F A ^ = A + E \widehat{A}=A+E A = A + E ν 2 / n \nu^2/n ν 2 / n A ^ \widehat{A} A ∥ A ^ k − A ∥ F ≤ ( 1 + O ( ν / σ k + 1 ( A ) ) ) ∥ A − A k ∥ F + O ( k ν ) \|\widehat{A}_k-A\|_F \leq (1+O(\sqrt{\nu/\sigma_{k+1}(A)}))\|A-A_k\|_F + O(\sqrt{k}\nu) ∥ A k − A ∥ F ≤ ( 1 + O ( ν / σ k + 1 ( A ) )) ∥ A − A k ∥ F + O ( k ν ) X 1 , ⋯ , X N ∼ N n ( 0 , A ) X_1,\cdots,X_N\sim\mathcal N_n(0,A) X 1 , ⋯ , X N ∼ N n ( 0 , A ) N = Ω ( n max { ϵ − 4 , k 2 } γ k ( A ) 2 log N ) N = \Omega\left(n\max\{\epsilon^{-4},k^2\}\gamma_k(A)^2\log N\right) N = Ω ( n max { ϵ − 4 , k 2 } γ k ( A ) 2 log N ) γ k ( A ) = σ 1 ( A ) / σ k + 1 ( A ) \gamma_k(A)=\sigma_1(A)/\sigma_{k+1}(A) γ k ( A ) = σ 1 ( A ) / σ k + 1 ( A ) ∥ A ^ k − A ∥ F ≤ ( 1 + O ( ϵ ) ) ∥ A − A k ∥ F \|\widehat{A}_k-A\|_F \leq (1+O(\epsilon))\|A-A_k\|_F ∥ A k − A ∥ F ≤ ( 1 + O ( ϵ )) ∥ A − A k ∥ F A ^ = 1 N ∑ i = 1 N X i X i ⊤ \widehat{A}=\frac{1}{N}\sum_{i=1}^N{X_iX_i^\top} A = N 1 ∑ i = 1 N X i X i ⊤
引用
@article{arxiv.1702.06861,
title = {On the Power of Truncated SVD for General High-rank Matrix Estimation Problems},
author = {Simon S. Du and Yining Wang and Aarti Singh},
journal= {arXiv preprint arXiv:1702.06861},
year = {2017}
}
备注
Accepted by NIPS 2017. Add gap-dependent bounds