Implicit Regularization in Matrix Factorization
Machine Learning
2017-05-26 v1 Machine Learning
Abstract
We study implicit regularization when optimizing an underdetermined quadratic objective over a matrix with gradient descent on a factorization of . We conjecture and provide empirical and theoretical evidence that with small enough step sizes and initialization close enough to the origin, gradient descent on a full dimensional factorization converges to the minimum nuclear norm solution.
Cite
@article{arxiv.1705.09280,
title = {Implicit Regularization in Matrix Factorization},
author = {Suriya Gunasekar and Blake Woodworth and Srinadh Bhojanapalli and Behnam Neyshabur and Nathan Srebro},
journal= {arXiv preprint arXiv:1705.09280},
year = {2017}
}