最有影响力子集选择：挑战、希望与超越
摘要
我们如何为机器学习模型的行为归因于其训练数据？虽然经典影响函数阐明了单个样本的影响，但往往无法捕捉到一组样本更复杂且更显著的集体影响。为解决这一挑战，我们研究了最有影响力子集选择(MISS)问题，该问题旨在识别对训练样本集合具有最大集体影响的子集。我们对MISS中主要方法进行了全面分析，阐明了其优势与不足。我们的发现表明，影响力基准的贪心启发式方法是MISS中占主导地位的算法类，甚至在线性回归中也可被证明会失败。我们描绘了失败模式，包括影响函数的误差以及集体影响的非可加结构。相反，我们证明了对这些启发式方法进行自适应版本的应用（即迭代地应用它们）可以有效地捕捉样本之间的相互作用，从而在一定程度上缓解了这些问题。在真实数据集上的实验证实了这些理论发现，并进一步表明自适应性的优势可以延伸到更复杂的情形，如分类任务和非线性神经网络。我们 conclusions our analysis by emphasizing the inherent trade-off between performance and computational efficiency, questioning the use of additive metrics such as the Linear Datamodeling Score, and offering a range of discussions. We conclude our analysis by emphasizing the inherent trade-off between performance and computational efficiency, questioning the use of additive metrics such as the Linear Datamodeling Score, and offering a range of discussions.
引用
@article{arxiv.2409.18153,
title = {Most Influential Subset Selection: Challenges, Promises, and Beyond},
author = {Yuzheng Hu and Pingbang Hu and Han Zhao and Jiaqi W. Ma},
journal= {arXiv preprint arXiv:2409.18153},
year = {2025}
}
备注
Accepted at the 38th Conference on Neural Information Processing Systems (NeurIPS 2024) Edit: Added discussion on a concurrent work