Predicting Generalization in Deep Learning via Local Measures of Distortion
Machine Learning
2020-12-17 v2 Machine Learning
Abstract
We study generalization in deep learning by appealing to complexity measures originally developed in approximation and information theory. While these concepts are challenged by the high-dimensional and data-defined nature of deep learning, we show that simple vector quantization approaches such as PCA, GMMs, and SVMs capture their spirit when applied layer-wise to deep extracted features giving rise to relatively inexpensive complexity measures that correlate well with generalization performance. We discuss our results in 2020 NeurIPS PGDL challenge.
Keywords
Cite
@article{arxiv.2012.06969,
title = {Predicting Generalization in Deep Learning via Local Measures of Distortion},
author = {Abhejit Rajagopal and Vamshi C. Madala and Shivkumar Chandrasekaran and Peder E. Z. Larson},
journal= {arXiv preprint arXiv:2012.06969},
year = {2020}
}
Comments
Added preprint footnote