An extended note on the multibin logarithmic score used in the FluSight competitions
Applications
2022-06-08 v1 Methodology
Abstract
In recent years the Centers for Disease Control and Prevention (CDC) have organized FluSight influenza forecasting competitions. To evaluate the participants' forecasts a multibin logarithmic score has been created, which is a non-standard variant of the established logarithmic score. Unlike the original log score, the multibin version is not proper and may thus encourage dishonest forecasting. We explore the practical consequences this may have, using forecasts from the 2016/17 FluSight competition for illustration.
Keywords
Cite
@article{arxiv.1910.07084,
title = {An extended note on the multibin logarithmic score used in the FluSight competitions},
author = {Johannes Bracher},
journal= {arXiv preprint arXiv:1910.07084},
year = {2022}
}
Comments
This note elaborates on a letter published in PNAS: Bracher, J: On the multibin logarithmic score used in the FluSight competitions. PNAS first published September 26, 2019 https://doi.org/10.1073/pnas.1912147116