The distance between the weights of the neural network is meaningful
Machine Learning
2021-02-03 v1
Abstract
In the application of neural networks, we need to select a suitable model based on the problem complexity and the dataset scale. To analyze the network's capacity, quantifying the information learned by the network is necessary. This paper proves that the distance between the neural network weights in different training stages can be used to estimate the information accumulated by the network in the training process directly. The experiment results verify the utility of this method. An application of this method related to the label corruption is shown at the end.
Keywords
Cite
@article{arxiv.2102.00396,
title = {The distance between the weights of the neural network is meaningful},
author = {Liqun Yang and Yijun Yang and Yao Wang and Zhenyu Yang and Wei Zeng},
journal= {arXiv preprint arXiv:2102.00396},
year = {2021}
}
Comments
10 pages, 13 figure