Adaptive Optimizer for Automated Hyperparameter Optimization Problem
Machine Learning
2022-01-31 v1 Optimization and Control
Abstract
The choices of hyperparameters have critical effects on the performance of machine learning models. In this paper, we present a general framework that is able to construct an adaptive optimizer, which automatically adjust the appropriate algorithm and parameters in the process of optimization. Examining the method of adaptive optimizer, we product an example of using genetic algorithm to construct an adaptive optimizer based on Bayesian Optimizer and compared effectiveness with original optimizer. Especially, It has great advantages in parallel optimization.
Cite
@article{arxiv.2201.12124,
title = {Adaptive Optimizer for Automated Hyperparameter Optimization Problem},
author = {Huayuan Sun},
journal= {arXiv preprint arXiv:2201.12124},
year = {2022}
}