English

The meta book and size-dependent properties of written language

Physics and Society 2009-12-10 v1 Computation and Language Data Analysis, Statistics and Probability

Abstract

Evidence is given for a systematic text-length dependence of the power-law index gamma of a single book. The estimated gamma values are consistent with a monotonic decrease from 2 to 1 with increasing length of a text. A direct connection to an extended Heap's law is explored. The infinite book limit is, as a consequence, proposed to be given by gamma = 1 instead of the value gamma=2 expected if the Zipf's law was ubiquitously applicable. In addition we explore the idea that the systematic text-length dependence can be described by a meta book concept, which is an abstract representation reflecting the word-frequency structure of a text. According to this concept the word-frequency distribution of a text, with a certain length written by a single author, has the same characteristics as a text of the same length pulled out from an imaginary complete infinite corpus written by the same author.

Keywords

Cite

@article{arxiv.0909.4385,
  title  = {The meta book and size-dependent properties of written language},
  author = {Sebastian Bernhardsson and Luis Enrique Correa da Rocha and Petter Minnhagen},
  journal= {arXiv preprint arXiv:0909.4385},
  year   = {2009}
}

Comments

7 pages, 6 figures, 1 table