On the nature of long-range letter correlations in texts
Computation and Language
2016-11-27 v1 Information Theory
math.IT
Abstract
The origin of long-range letter correlations in natural texts is studied using random walk analysis and Jensen-Shannon divergence. It is concluded that they result from slow variations in letter frequency distribution, which are a consequence of slow variations in lexical composition within the text. These correlations are preserved by random letter shuffling within a moving window. As such, they do reflect structural properties of the text, but in a very indirect manner.
Keywords
Cite
@article{arxiv.0809.0103,
title = {On the nature of long-range letter correlations in texts},
author = {Dmitrii Y. Manin},
journal= {arXiv preprint arXiv:0809.0103},
year = {2016}
}
Comments
14 pages, 5 figures, unpublished