Active Learning under Label Shift
Abstract
We address the problem of active learning under label shift: when the class proportions of source and target domains differ. We introduce a "medial distribution" to incorporate a tradeoff between importance weighting and class-balanced sampling and propose their combined usage in active learning. Our method is known as Mediated Active Learning under Label Shift (MALLS). It balances the bias from class-balanced sampling and the variance from importance weighting. We prove sample complexity and generalization guarantees for MALLS which show active learning reduces asymptotic sample complexity even under arbitrary label shift. We empirically demonstrate MALLS scales to high-dimensional datasets and can reduce the sample complexity of active learning by 60% in deep active learning tasks.
Cite
@article{arxiv.2007.08479,
title = {Active Learning under Label Shift},
author = {Eric Zhao and Anqi Liu and Animashree Anandkumar and Yisong Yue},
journal= {arXiv preprint arXiv:2007.08479},
year = {2021}
}
Comments
18 pages, 9 figures, to appear at the 2021 International Conference on Artificial Intelligence and Statistics (AIStats)