Related papers: Zipf's law unzipped
Order statistics find applications in various areas of communications and signal processing. In this paper, we introduce an unified analytical framework to determine the joint statistics of partial sums of ordered random variables (RVs).…
This paper studies the limits of language models' statistical learning in the context of Zipf's law. First, we demonstrate that Zipf-law token distribution emerges irrespective of the chosen tokenization. Second, we show that Zipf…
A solution for the time- and age-dependent connectivity distribution of a growing random network is presented. The network is built by adding sites which link to earlier sites with a probability A_k which depends on the number of…
This article presents a concise proof of the famous Benford's law when the distribution has a Riemann integrable probability density function and provides a criterion to judge whether a distribution obeys the law. The proof is intuitive and…
The Chapter starts with introductory information about quantitative linguistics notions, like rank--frequency dependence, Zipf's law, frequency spectra, etc. Similarities in distributions of words in texts with level occupation in quantum…
In this article, the relationship between two well-accepted empirical propositions regarding the distribution of population in cities, namely, Gibrat's law and Zipf's law, are rigorously examined using the Chinese census data. Our findings…
Apparently random events in nature often reveal hidden patterns when analysed using diverse and robust statistical tools. Power-law distributions, for example, project diverse natural phenomenon, ranging from earthquakes1 to heartbeat…
We use the formalism of 'Maximum Principle of Shannon's Entropy' to derive the general power law distribution function, using what seems to be a reasonable physical assumption, namely, the demand of a constant mean "internal order"…
We present an impossibility result, called a theorem about facts and words, which pertains to a general communication system. The theorem states that the number of distinct words used in a finite text is roughly greater than the number of…
We show that statistical criticality, i.e. the occurrence of power law frequency distributions, arises in samples that are maximally informative about the underlying generating process. In order to reach this conclusion, we first identify…
Random geometric graphs (RGG) can be formalized as hidden-variables models where the hidden variables are the coordinates of the nodes. Here we develop a general approach to extract the typical configurations of a generic hidden-variables…
We investigate the possible origin of hierarchical structures in complex systems describable in terms of a finite and small number of parameters which control the behavioral pattern at each level of organization. We argue that the…
The science of cities seeks to understand and explain regularities observed in the world's major urban systems. Modelling the population evolution of cities is at the core of this science and of all urban studies. Quantitatively, the most…
In his pioneering research, G. K. Zipf formulated a couple of statistical laws on the relationship between the frequency of a word with its number of meanings: the law of meaning distribution, relating the frequency of a word and its…
Learning is a distinctive feature of intelligent behaviour. High-throughput experimental data and Big Data promise to open new windows on complex systems such as cells, the brain or our societies. Yet, the puzzling success of Artificial…
In many complex systems, for the activity f(i) of the constituents or nodes i, a power-law relationship was discovered between the standard deviation sigma(i) and the average strength of the activity: sigma(i) ~ <f(i)>^alpha; universal…
Tokenization is a fundamental step in natural language processing (NLP) and other sequence modeling domains, where the choice of vocabulary size significantly impacts model performance. Despite its importance, selecting an optimal…
Stochastic equations constitute a major ingredient in many branches of science, from physics to biology and engineering. Not surprisingly, they appear in many quantitative studies of complex systems. In particular, this type of equation is…
The probability that $m$ randomly chosen elements of a finite power associative loop $C$ have prescribed orders and generate $C$ is calculated in terms of certain constants related to the action of $Aut(C)$ on the subloop lattice of $C$. As…
Stopwords are words that are not very informative to the content or the meaning of a language text. Most stopwords are function words but can also be common verbs, adjectives and adverbs. In contrast to the well known Zipf's law for…