Not All Lotteries Are Made Equal

Surya Kant Sahu; Sai Mitheran; Somya Suhans Mahapatra

Not All Lotteries Are Made Equal

Machine Learning 2022-06-17 v1

Authors: Surya Kant Sahu , Sai Mitheran , Somya Suhans Mahapatra

Abstract

The Lottery Ticket Hypothesis (LTH) states that for a reasonably sized neural network, a sub-network within the same network yields no less performance than the dense counterpart when trained from the same initialization. This work investigates the relation between model size and the ease of finding these sparse sub-networks. We show through experiments that, surprisingly, under a finite budget, smaller models benefit more from Ticket Search (TS).

Cite

@article{arxiv.2206.08175,
  title  = {Not All Lotteries Are Made Equal},
  author = {Surya Kant Sahu and Sai Mitheran and Somya Suhans Mahapatra},
  journal= {arXiv preprint arXiv:2206.08175},
  year   = {2022}
}

Comments

Accepted at ICML 2022 HAET Workshop

Related papers

View all related →