REMIND Your Neural Network to Prevent Catastrophic Forgetting
Abstract
People learn throughout life. However, incrementally updating conventional neural networks leads to catastrophic forgetting. A common remedy is replay, which is inspired by how the brain consolidates memory. Replay involves fine-tuning a network on a mixture of new and old instances. While there is neuroscientific evidence that the brain replays compressed memories, existing methods for convolutional networks replay raw images. Here, we propose REMIND, a brain-inspired approach that enables efficient replay with compressed representations. REMIND is trained in an online manner, meaning it learns one example at a time, which is closer to how humans learn. Under the same constraints, REMIND outperforms other methods for incremental class learning on the ImageNet ILSVRC-2012 dataset. We probe REMIND's robustness to data ordering schemes known to induce catastrophic forgetting. We demonstrate REMIND's generality by pioneering online learning for Visual Question Answering (VQA).
Cite
@article{arxiv.1910.02509,
title = {REMIND Your Neural Network to Prevent Catastrophic Forgetting},
author = {Tyler L. Hayes and Kushal Kafle and Robik Shrestha and Manoj Acharya and Christopher Kanan},
journal= {arXiv preprint arXiv:1910.02509},
year = {2020}
}
Comments
To appear in the European Conference on Computer Vision (ECCV-2020)