When hyperparameter optimization of a machine learning algorithm is repeated for multiple datasets it is possible to transfer knowledge to an optimization run on a new dataset. We develop a new hyperparameter-free ensemble model for Bayesian optimization that is a generalization of two existing transfer learning extensions to Bayesian optimization and establish a worst-case bound compared to vanilla Bayesian optimization. Using a large collection of hyperparameter optimization benchmark problems, we demonstrate that our contributions substantially reduce optimization time compared to standard Gaussian process-based Bayesian optimization and improve over the current state-of-the-art for transfer hyperparameter optimization.
@article{arxiv.1802.02219,
title = {Practical Transfer Learning for Bayesian Optimization},
author = {Matthias Feurer and Benjamin Letham and Frank Hutter and Eytan Bakshy},
journal= {arXiv preprint arXiv:1802.02219},
year = {2022}
}
Comments
This version fixes a minor error in the equation in Section 3.2 of V3