Novelty Driven Evolutionary Neural Architecture Search
Abstract
Evolutionary algorithms (EA) based neural architecture search (NAS) involves evaluating each architecture by training it from scratch, which is extremely time-consuming. This can be reduced by using a supernet for estimating the fitness of an architecture due to weight sharing among all architectures in the search space. However, the estimated fitness is very noisy due to the co-adaptation of the operations in the supernet which results in NAS methods getting trapped in local optimum. In this paper, we propose a method called NEvoNAS wherein the NAS problem is posed as a multi-objective problem with 2 objectives: (i) maximize architecture novelty, (ii) maximize architecture fitness/accuracy. The novelty search is used for maintaining a diverse set of solutions at each generation which helps avoiding local optimum traps while the architecture fitness is calculated using supernet. NSGA-II is used for finding the \textit{pareto optimal front} for the NAS problem and the best architecture in the pareto front is returned as the searched architecture. Exerimentally, NEvoNAS gives better results on 2 different search spaces while using significantly less computational resources as compared to previous EA-based methods. The code for our paper can be found in https://github.com/nightstorm0909/NEvoNAS.
Cite
@article{arxiv.2204.00188,
title = {Novelty Driven Evolutionary Neural Architecture Search},
author = {Nilotpal Sinha and Kuan-Wen Chen},
journal= {arXiv preprint arXiv:2204.00188},
year = {2022}
}
Comments
Accepted as poster in GECCO 2022. arXiv admin note: substantial text overlap with arXiv:2107.07266, arXiv:2203.01559