Universal Training of Neural Networks to Achieve Bayes Optimal Classification Accuracy
Abstract
This work invokes the notion of -divergence to introduce a novel upper bound on the Bayes error rate of a general classification task. We show that the proposed bound can be computed by sampling from the output of a parameterized model. Using this practical interpretation, we introduce the Bayes optimal learning threshold (BOLT) loss whose minimization enforces a classification model to achieve the Bayes error rate. We validate the proposed loss for image and text classification tasks, considering MNIST, Fashion-MNIST, CIFAR-10, and IMDb datasets. Numerical experiments demonstrate that models trained with BOLT achieve performance on par with or exceeding that of cross-entropy, particularly on challenging datasets. This highlights the potential of BOLT in improving generalization.
Cite
@article{arxiv.2501.07754,
title = {Universal Training of Neural Networks to Achieve Bayes Optimal Classification Accuracy},
author = {Mohammadreza Tavasoli Naeini and Ali Bereyhi and Morteza Noshad and Ben Liang and Alfred O. Hero},
journal= {arXiv preprint arXiv:2501.07754},
year = {2025}
}
Comments
Accepted to ICASSP 2025