Robust importance-weighted cross-validation under sample selection bias
Machine Learning
2019-08-28 v3 Machine Learning
Abstract
Cross-validation under sample selection bias can, in principle, be done by importance-weighting the empirical risk. However, the importance-weighted risk estimator produces sub-optimal hyperparameter estimates in problem settings where large weights arise with high probability. We study its sampling variance as a function of the training data distribution and introduce a control variate to increase its robustness to problematically large weights.
Keywords
Cite
@article{arxiv.1710.06514,
title = {Robust importance-weighted cross-validation under sample selection bias},
author = {Wouter M. Kouw and Jesse H. Krijthe and Marco Loog},
journal= {arXiv preprint arXiv:1710.06514},
year = {2019}
}
Comments
6 pages, 8 figures, Accepted to the IEEE International Workshop on Machine Learning for Signal Processing 2019