The key of the out-of-distribution (OOD) generalization is to generalize invariance from training domains to target domains. The variance risk extrapolation (V-REx) is a practical OOD method, which depends on a domain-level regularization but lacks theoretical verifications about its motivation and utility. This article provides theoretical insights into V-REx by studying a variance-based regularizer. We propose Risk Variance Penalization (RVP), which slightly changes the regularization of V-REx but addresses the theory concerns about V-REx. We provide theoretical explanations and a theory-inspired tuning scheme for the regularization parameter of RVP. Our results point out that RVP discovers a robust predictor. Finally, we experimentally show that the proposed regularizer can find an invariant predictor under certain conditions.
Cite
@article{arxiv.2006.07544,
title = {Risk Variance Penalization},
author = {Chuanlong Xie and Haotian Ye and Fei Chen and Yue Liu and Rui Sun and Zhenguo Li},
journal= {arXiv preprint arXiv:2006.07544},
year = {2021}
}