English

A Crucial Parameter for Rank-Frequency Relation in Natural Languages

Computation and Language 2024-02-02 v1

Abstract

frα(r+γ)βf \propto r^{-\alpha} \cdot (r+\gamma)^{-\beta} has been empirically shown more precise than a na\"ive power law frαf\propto r^{-\alpha} to model the rank-frequency (rr-ff) relation of words in natural languages. This work shows that the only crucial parameter in the formulation is γ\gamma, which depicts the resistance to vocabulary growth on a corpus. A method of parameter estimation by searching an optimal γ\gamma is proposed, where a ``zeroth word'' is introduced technically for the calculation. The formulation and parameters are further discussed with several case studies.

Keywords

Cite

@article{arxiv.2402.00271,
  title  = {A Crucial Parameter for Rank-Frequency Relation in Natural Languages},
  author = {Chenchen Ding},
  journal= {arXiv preprint arXiv:2402.00271},
  year   = {2024}
}
R2 v1 2026-06-28T14:33:58.602Z