Lottery Tickets in Linear Models: An Analysis of Iterative Magnitude Pruning

Bryn Elesedy; Varun Kanade; Yee Whye Teh

Lottery Tickets in Linear Models: An Analysis of Iterative Magnitude Pruning

Machine Learning 2021-07-06 v3 Machine Learning

Authors: Bryn Elesedy , Varun Kanade , Yee Whye Teh

Abstract

We analyse the pruning procedure behind the lottery ticket hypothesis arXiv:1803.03635v5, iterative magnitude pruning (IMP), when applied to linear models trained by gradient flow. We begin by presenting sufficient conditions on the statistical structure of the features under which IMP prunes those features that have smallest projection onto the data. Following this, we explore IMP as a method for sparse estimation.

Keywords

lottery ticket hypothesis iterative magnitude pruning linear models gradient flow feature projection sparse estimation

Cite

@article{arxiv.2007.08243,
  title  = {Lottery Tickets in Linear Models: An Analysis of Iterative Magnitude Pruning},
  author = {Bryn Elesedy and Varun Kanade and Yee Whye Teh},
  journal= {arXiv preprint arXiv:2007.08243},
  year   = {2021}
}

Comments

Updated for Sparsity in Neural Networks Workshop

Related papers

View all related →