Comparison of Bayesian and particle swarm algorithms for hyperparameter optimisation in machine learning applications in high energy physics
Abstract
When using machine learning (ML) techniques, users typically need to choose a plethora of algorithm-specific parameters, referred to as hyperparameters. In this paper, we compare the performance of two algorithms, particle swarm optimisation (PSO) and Bayesian optimisation (BO), for the autonomous determination of these hyperparameters in applications to different ML tasks typical for the field of high energy physics (HEP). Our evaluation of the performance includes a comparison of the capability of the PSO and BO algorithms to make efficient use of the highly parallel computing resources that are characteristic of contemporary HEP experiments.
Keywords
Cite
@article{arxiv.2201.06809,
title = {Comparison of Bayesian and particle swarm algorithms for hyperparameter optimisation in machine learning applications in high energy physics},
author = {Laurits Tani and Christian Veelken},
journal= {arXiv preprint arXiv:2201.06809},
year = {2023}
}
Comments
Accepted by Computer Physics Communications. Changes made compared to previous version: added references to other strategies, added Zenodo entry for the implemented software, added a brief description of PSO, added more explanations regarding the benchmark tasks