利用拓扑数据分析预测神经网络的泛化差距
机器学习
2023-08-15 v2 代数拓扑
摘要
理解神经网络在未见数据上的泛化能力对于设计更鲁棒、更可靠的模型至关重要。本文利用拓扑数据分析的方法研究神经网络的泛化差距。为此，我们在训练阶段后根据神经元激活相关性构建加权图，并计算其同调持续图（homological persistence diagrams），旨在捕捉与网络泛化能力相关的模式。我们比较了持续图中不同数值摘要的效用，表明其中部分摘要的组合能够在无需测试集的情况下准确预测并部分解释泛化差距。在两个计算机视觉识别任务（CIFAR10 和 SVHN）上的评估表明，与最先进（state-of-the-art）方法相比，本文方法在泛化差距预测上具有竞争力。
引用
@article{arxiv.2203.12330,
title = {Predicting the generalization gap in neural networks using topological data analysis},
author = {Rubén Ballester and Xavier Arnal Clemente and Carles Casacuberta and Meysam Madadi and Ciprian A. Corneanu and Sergio Escalera},
journal= {arXiv preprint arXiv:2203.12330},
year = {2023}
}
备注
24 pages, 7 figures. The Related Work section has been updated and the experiments have been executed anew including a 5x2-fold cross-validation scheme. Figure 4.3 has been crucially improved thanks to the discovery that the clusters of neural networks that appear in that figure correspond to different depths of the corresponding architectures