关于用于遗忘神经网络的牛顿方法

随着在个人数据上训练的神经网络（NN）的广泛应用，机器遗忘对于使个体能够行使其个人数据所有权，特别是从训练好的NN中实现"被遗忘权"，变得越来越重要。由于重新训练计算昂贵，我们寻求一种近似遗忘算法，用于NN，该算法返回与重新训练的oracle相同的模型。虽然牛顿方法已成功用于approximately unlearn linear models, we observe that adapting it for NN is challenging due to degenerate Hessians that make computing Newton's update impossible. Additionally, we show that when coupled with popular techniques to resolve the degeneracy, Newton's method often incurs offensively large norm updates and empirically degrades model performance post-unlearning. To address these challenges, we propose CureNewton's method, a principle approach that leverages cubic regularization to handle the Hessian degeneracy effectively. The added regularizer eliminates the need for manual finetuning and affords a natural interpretation within the unlearning context. Experiments across different models and datasets show that our method can achieve competitive unlearning performance to the state-of-the-art algorithm in practical unlearning settings, while being theoretically justified and efficient in running time.