We propose a novel approach to achieving invariance for deep neural networks in the form of inducing amnesia to unwanted factors of data through a new adversarial forgetting mechanism. We show that the forgetting mechanism serves as an information-bottleneck, which is manipulated by the adversarial training to learn invariance to unwanted factors. Empirical results show that the proposed framework achieves state-of-the-art performance at learning invariance in both nuisance and bias settings on a diverse collection of datasets and tasks.
@article{arxiv.1911.04060,
title = {Invariant Representations through Adversarial Forgetting},
author = {Ayush Jaiswal and Daniel Moyer and Greg Ver Steeg and Wael AbdAlmageed and Premkumar Natarajan},
journal= {arXiv preprint arXiv:1911.04060},
year = {2019}
}
Comments
To appear in Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI-20)