Long-Tail Theory under Gaussian Mixtures
Machine Learning
2023-07-26 v2 Machine Learning
Abstract
We suggest a simple Gaussian mixture model for data generation that complies with Feldman's long tail theory (2020). We demonstrate that a linear classifier cannot decrease the generalization error below a certain level in the proposed model, whereas a nonlinear classifier with a memorization capacity can. This confirms that for long-tailed distributions, rare training examples must be considered for optimal generalization to new data. Finally, we show that the performance gap between linear and nonlinear models can be lessened as the tail becomes shorter in the subpopulation frequency distribution, as confirmed by experiments on synthetic and real data.
Keywords
Cite
@article{arxiv.2307.10736,
title = {Long-Tail Theory under Gaussian Mixtures},
author = {Arman Bolatov and Maxat Tezekbayev and Igor Melnykov and Artur Pak and Vassilina Nikoulina and Zhenisbek Assylbekov},
journal= {arXiv preprint arXiv:2307.10736},
year = {2023}
}
Comments
accepted to ECAI 2023