English

Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Search

Machine Learning 2021-04-13 v1 Computer Vision and Pattern Recognition

Abstract

Weight sharing has become a de facto standard in neural architecture search because it enables the search to be done on commodity hardware. However, recent works have empirically shown a ranking disorder between the performance of stand-alone architectures and that of the corresponding shared-weight networks. This violates the main assumption of weight-sharing NAS algorithms, thus limiting their effectiveness. We tackle this issue by proposing a regularization term that aims to maximize the correlation between the performance rankings of the shared-weight network and that of the standalone architectures using a small set of landmark architectures. We incorporate our regularization term into three different NAS algorithms and show that it consistently improves performance across algorithms, search-spaces, and tasks.

Keywords

Cite

@article{arxiv.2104.05309,
  title  = {Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Search},
  author = {Kaicheng Yu and Rene Ranftl and Mathieu Salzmann},
  journal= {arXiv preprint arXiv:2104.05309},
  year   = {2021}
}

Comments

Accepted to CVPR 2021

R2 v1 2026-06-24T01:04:16.196Z