English

What killed the Convex Booster ?

Machine Learning 2022-05-26 v2

Abstract

A landmark negative result of Long and Servedio established a worst-case spectacular failure of a supervised learning trio (loss, algorithm, model) otherwise praised for its high precision machinery. Hundreds of papers followed up on the two suspected culprits: the loss (for being convex) and/or the algorithm (for fitting a classical boosting blueprint). Here, we call to the half-century+ founding theory of losses for class probability estimation (properness), an extension of Long and Servedio's results and a new general boosting algorithm to demonstrate that the real culprit in their specific context was in fact the (linear) model class. We advocate for a more general stanpoint on the problem as we argue that the source of the negative result lies in the dark side of a pervasive -- and otherwise prized -- aspect of ML: \textit{parameterisation}.

Cite

@article{arxiv.2205.09628,
  title  = {What killed the Convex Booster ?},
  author = {Yishay Mansour and Richard Nock and Robert C. Williamson},
  journal= {arXiv preprint arXiv:2205.09628},
  year   = {2022}
}
R2 v1 2026-06-24T11:22:26.853Z