English

Correct ordering in the Zipf-Poisson ensemble

Methodology 2011-01-14 v1 Statistics Theory Statistics Theory

Abstract

We consider a Zipf--Poisson ensemble in which Xi\poi(Niα)X_i\sim\poi(Ni^{-\alpha}) for α>1\alpha>1 and N>0N>0 and integers i1i\ge 1. As NN\to\infty the first n(N)n'(N) random variables have their proper order X1>X2>...>XnX_1>X_2>...>X_{n'} relative to each other, with probability tending to 1 for nn' up to (AN/log(N))1/(α+2)(AN/\log(N))^{1/(\alpha+2)} for an explicit constant A(α)3/4A(\alpha)\ge 3/4. The rate N1/(α+2)N^{1/(\alpha+2)} cannot be achieved. The ordering of the first n(N)n'(N) entities does not preclude Xm>XnX_m>X_{n'} for some interloping m>nm>n'. The first n"n" random variables are correctly ordered exclusive of any interlopers, with probability tending to 1 if n"(BN/log(N))1/(α+2)n"\le (BN/\log(N))^{1/(\alpha+2)} for B<AB<A. For a Zipf--Poisson model of the British National Corpus, which has a total word count of 100,000,000100{,}000{,}000, our result estimates that the 72 words with the highest counts are properly ordered.

Cite

@article{arxiv.1101.2481,
  title  = {Correct ordering in the Zipf-Poisson ensemble},
  author = {Justin S. Dyer and Art B. Owen},
  journal= {arXiv preprint arXiv:1101.2481},
  year   = {2011}
}
R2 v1 2026-06-21T17:11:18.754Z