English

Factorized AdaBoost.MH Achieves the Same Convergence Rate as AdaBoost.MH

Machine Learning 2026-08-02 v1

Abstract

AdaBoost.MH reduces multi-class classification to a collection of binary subproblems and enjoys the classical boosting-type convergence guarantee under a weak learning condition. A more structured variant, Factorized AdaBoost.MH, uses base classifiers of the form h(x)=αvφ(x)\mathbf{h}(x)=\alpha \mathbf{v} \bm{\varphi}(x), where a single binary classifier φ\bm{\varphi} is shared across all classes and the label dependence is carried by a vote vector v{±1}K\mathbf{v} \in\{\pm1\}^K. This factorization is algorithmically attractive and achieves better performance in practice, but its convergence depends on whether one can always choose a vote vector with sufficiently large induced binary weight mass. Previous work resolved this question with a lower bound max{1/n,1/2K}\max\{1/n,1/\sqrt{2K}\}, which still leaves a dimension-dependent slowdown relative to the original AdaBoost.MH analysis. In this paper, we sharpen this combinatorial step. For the minimax quantity Wn,K\mathfrak{W}_{n,K} governing the factorized edge, we prove max{1/n,CK}Wn,KCmin{n,K}\max\{1/n,C_K\}\le\mathfrak{W}_{n,K}\le C_{\min\{n,K\}}, where Cq=1C_q=1 for q=1q=1, Cq=q/(3q4)C_q=q/(3q-4) for even q2q\ge2, and Cq=(q+1)/(3q1)C_q=(q+1)/(3q-1) for odd q2q\ge2. Since Cq1/3C_q\downarrow 1/3, our bounds show that Wn,K=Θ(1)\mathfrak{W}_{n,K}=\Theta(1) uniformly over nn and KK. Consequently, Factorized AdaBoost.MH achieves the same boosting-type convergence rate as AdaBoost.MH up to a universal constant factor, removing the previously suggested additional dependence on nn or KK in the number of boosting rounds.

Cite

@article{arxiv.2608.01091,
  title  = {Factorized AdaBoost.MH Achieves the Same Convergence Rate as AdaBoost.MH},
  author = {Xin Zou and Jingyuan Xu},
  journal= {arXiv preprint arXiv:2608.01091},
  year   = {2026}
}