Extrapolating Expected Accuracies for Large Multi-Class Problems
Machine Learning
2017-12-29 v1 Computer Vision and Pattern Recognition
Machine Learning
Abstract
The difficulty of multi-class classification generally increases with the number of classes. Using data from a subset of the classes, can we predict how well a classifier will scale with an increased number of classes? Under the assumptions that the classes are sampled identically and independently from a population, and that the classifier is based on independently learned scoring functions, we show that the expected accuracy when the classifier is trained on k classes is the (k-1)st moment of a certain distribution that can be estimated from data. We present an unbiased estimation method based on the theory, and demonstrate its application on a facial recognition example.
Cite
@article{arxiv.1712.09713,
title = {Extrapolating Expected Accuracies for Large Multi-Class Problems},
author = {Charles Zheng and Rakesh Achanta and Yuval Benjamini},
journal= {arXiv preprint arXiv:1712.09713},
year = {2017}
}
Comments
Submitted to JMLR