Machine unlearning aims to revoke some training data after learning in response to requests from users, model developers, and administrators. Most previous methods are based on direct fine-tuning, which may neither remove data completely nor retain full performances on the remain data. In this work, we find that, by first masking some important parameters before fine-tuning, the performances of unlearning could be significantly improved. We propose a new masking strategy tailored to unlearning based on Fisher information. Experiments on various datasets and network structures show the effectiveness of the method: without any fine-tuning, the proposed Fisher masking could unlearn almost completely while maintaining most of the performance on the remain data. It also exhibits stronger stability compared to other unlearning baselines
@article{arxiv.2310.05331,
title = {Unlearning with Fisher Masking},
author = {Yufang Liu and Changzhi Sun and Yuanbin Wu and Aimin Zhou},
journal= {arXiv preprint arXiv:2310.05331},
year = {2023}
}