Learning Morphisms with Gauss-Newton Approximation for Growing Networks
Abstract
A popular method for Neural Architecture Search (NAS) is based on growing networks via small local changes to the network's architecture called network morphisms. These methods start with a small seed network and progressively grow the network by adding new neurons in an automated way. However, it remains a challenge to efficiently determine which parts of the network are best to grow. Here we propose a NAS method for growing a network by using a Gauss-Newton approximation of the loss function to efficiently learn and evaluate candidate network morphisms. We compare our method with state of the art NAS methods for CIFAR-10 and CIFAR-100 classification tasks, and conclude our method learns similar quality or better architectures at a smaller computational cost.
Cite
@article{arxiv.2411.05855,
title = {Learning Morphisms with Gauss-Newton Approximation for Growing Networks},
author = {Neal Lawton and Aram Galstyan and Greg Ver Steeg},
journal= {arXiv preprint arXiv:2411.05855},
year = {2024}
}
Comments
12 pages, 4 figures