Sharpening Occam's Razor
Machine Learning
2009-09-29 v2 Disordered Systems and Neural Networks
Artificial Intelligence
Computational Complexity
Probability
Data Analysis, Statistics and Probability
Abstract
We provide a new representation-independent formulation of Occam's razor theorem, based on Kolmogorov complexity. This new formulation allows us to: (i) Obtain better sample complexity than both length-based and VC-based versions of Occam's razor theorem, in many applications. (ii) Achieve a sharper reverse of Occam's razor theorem than previous work. Specifically, we weaken the assumptions made in an earlier publication, and extend the reverse to superpolynomial running times.
Cite
@article{arxiv.cs/0201005,
title = {Sharpening Occam's Razor},
author = {Ming Li and John Tromp and Paul Vitanyi},
journal= {arXiv preprint arXiv:cs/0201005},
year = {2009}
}
Comments
LaTeX 13 pages; Proc 8th COCOON, LNCS 2387, Springer-Verlag, Berlin, 2002, 411--419