Empirical study of extreme overfitting points of neural networks
Machine Learning
2020-04-03 v2 Machine Learning
Abstract
In this paper we propose a method of obtaining points of extreme overfitting - parameters of modern neural networks, at which they demonstrate close to 100 % training accuracy, simultaneously with almost zero accuracy on the test sample. Despite the widespread opinion that the overwhelming majority of critical points of the loss function of a neural network have equally good generalizing ability, such points have a huge generalization error. The paper studies the properties of such points and their location on the surface of the loss function of modern neural networks.
Keywords
Cite
@article{arxiv.1906.06295,
title = {Empirical study of extreme overfitting points of neural networks},
author = {Daniil Merkulov and Ivan Oseledets},
journal= {arXiv preprint arXiv:1906.06295},
year = {2020}
}