English

Conserving Fuel in Statistical Language Learning: Predicting Data Requirements

cmp-lg 2008-02-03 v1 Computation and Language

Abstract

In this paper I address the practical concern of predicting how much training data is sufficient for a statistical language learning system. First, I briefly review earlier results and show how these can be combined to bound the expected accuracy of a mode-based learner as a function of the volume of training data. I then develop a more accurate estimate of the expected accuracy function under the assumption that inputs are uniformly distributed. Since this estimate is expensive to compute, I also give a close but cheaply computable approximation to it. Finally, I report on a series of simulations exploring the effects of inputs that are not uniformly distributed. Although these results are based on simplistic assumptions, they are a tentative step toward a useful theory of data requirements for SLL systems.

Keywords

Cite

@article{arxiv.cmp-lg/9509002,
  title  = {Conserving Fuel in Statistical Language Learning: Predicting Data Requirements},
  author = {Mark Lauer},
  journal= {arXiv preprint arXiv:cmp-lg/9509002},
  year   = {2008}
}

Comments

8 pages