Sample compression schemes for VC classes
Abstract
Sample compression schemes were defined by Littlestone and Warmuth (1986) as an abstraction of the structure underlying many learning algorithms. Roughly speaking, a sample compression scheme of size means that given an arbitrary list of labeled examples, one can retain only of them in a way that allows to recover the labels of all other examples in the list. They showed that compression implies PAC learnability for binary-labeled classes, and asked whether the other direction holds. We answer their question and show that every concept class with VC dimension has a sample compression scheme of size exponential in . The proof uses an approximate minimax phenomenon for binary matrices of low VC dimension, which may be of interest in the context of game theory.
Keywords
Cite
@article{arxiv.1503.06960,
title = {Sample compression schemes for VC classes},
author = {Shay Moran and Amir Yehudayoff},
journal= {arXiv preprint arXiv:1503.06960},
year = {2015}
}
Comments
14 pages. The previous version of this text contained an error; Theorem 2.1 in it is false. This error only affects the statement for multi-labeled classes, and the construction for binary-labeled classes still holds. In the new version of the text, we added a relevant discussion in Section 4