Large deep neural networks are powerful, but exhibit undesirable behaviors such as memorization and sensitivity to adversarial examples. In this work, we propose mixup, a simple learning principle to alleviate these issues. In essence, mixup trains a neural network on convex combinations of pairs of examples and their labels. By doing so, mixup regularizes the neural network to favor simple linear behavior in-between training examples. Our experiments on the ImageNet-2012, CIFAR-10, CIFAR-100, Google commands and UCI datasets show that mixup improves the generalization of state-of-the-art neural network architectures. We also find that mixup reduces the memorization of corrupt labels, increases the robustness to adversarial examples, and stabilizes the training of generative adversarial networks.
@article{arxiv.1710.09412,
title = {mixup: Beyond Empirical Risk Minimization},
author = {Hongyi Zhang and Moustapha Cisse and Yann N. Dauphin and David Lopez-Paz},
journal= {arXiv preprint arXiv:1710.09412},
year = {2018}
}
Comments
ICLR camera ready version. Changes vs V1: fix repo URL; add ablation studies; add mixup + dropout etc