A Short Introduction to Model Selection, Kolmogorov Complexity and Minimum Description Length (MDL)
Machine Learning
2010-05-17 v2 Computational Complexity
Abstract
The concept of overfitting in model selection is explained and demonstrated with an example. After providing some background information on information theory and Kolmogorov complexity, we provide a short explanation of Minimum Description Length and error minimization. We conclude with a discussion of the typical features of overfitting in model selection.
Cite
@article{arxiv.1005.2364,
title = {A Short Introduction to Model Selection, Kolmogorov Complexity and Minimum Description Length (MDL)},
author = {Volker Nannen},
journal= {arXiv preprint arXiv:1005.2364},
year = {2010}
}
Comments
20 pages, Chapter 1 of The Paradox of Overfitting, Master's thesis, Rijksuniversiteit Groningen, 2003