Related papers: Zipf's law, power laws, and maximum entropy
A plethora of natural and socio-economic phenomena share a striking statistical regularity, that is the magnitude of elements decreases with a power law as a function of their position in a ranking of magnitude. Such regularity is known as…
Zipf's law is well known in linguistics: the frequency of a word is inversely proportional to its rank. This is a special case of a more general power law, a common phenomenon in many kinds of real-world statistical data. Here, it is shown…
Here we present a new class of optimality for coding systems. Members of that class are displaced linearly from optimal coding and thus exhibit Zipf's law, namely a power-law distribution of frequency ranks. Within that class, Zipf's law,…
In his pioneering research, G. K. Zipf observed that more frequent words tend to have more meanings, and showed that the number of meanings of a word grows as the square root of its frequency. He derived this relationship from two…
The principle of maximum entropy provides a useful method for inferring statistical mechanics models from observations in correlated systems, and is widely used in a variety of fields where accurate data are available. While the assumptions…
A new angle of view is proposed to find the simple rules dominating complex systems and regular patterns behind random phenomena such as cities. Hierarchy of cities reflects the ubiquitous structure frequently observed in the natural world…
The fundamentals of the Maximum Entropy principle as a rule for assigning and updating probabilities are revisited. The Shannon-Jaynes relative entropy is vindicated as the optimal criterion for use with an updating rule. A constructive…
We study a deliberately simple, fully non-linguistic model of text: a sequence of independent draws from a finite alphabet of letters plus a single space symbol. A word is defined as a maximal block of non-space symbols. Within this…
An expression is proposed for determining the error caused on entropy estimates by finite sample effects. This expression is based on the Ansatz that the ranked distribution of probabilities tends to follow an empirical Zipf law.
Most of various large-size complex systems in nature and society can be well described as complex networks (graphs) to better understand the evolutional mechanisms and dynamical functions behind themselves. Of some part follow scale-free…
Calculating the Shannon entropy for symbolic sequences has been widely considered in many fields. For descriptive statistical problems such as estimating the N-gram entropy of English language text, a common approach is to use as much data…
Bayesian modelling and statistical text analysis rely on informed probability priors to encourage good solutions. This paper empirically analyses whether text in medical discharge reports follow Zipf's law, a commonly assumed statistical…
In a language corpus, the probability that a word occurs $n$ times is often proportional to $1/n^2$. Assigning rank, $s$, to words according to their abundance, $\log s$ vs $\log n$ typically has a slope of minus one. That simple Zipf's law…
It has been shown recently that a specific class of path-dependent stochastic processes, which reduce their sample space as they unfold, lead to exact scaling laws in frequency and rank distributions. Such Sample Space Reducing processes…
We shall prove that the celebrated R\'enyi entropy is the first example of a new family of infinitely many multi-parametric entropies. We shall call them the $Z$-entropies. Each of them, under suitable hypotheses, generalizes the celebrated…
Information entropy and Zipf's law in the field of information theory have been used for studying the disassembly of nuclei in the framework of the isospin dependent lattice gas model and molecular dynamical model. We found that the…
Zipf's law is a fundamental paradigm in the statistics of written and spoken natural language as well as in other communication systems. We raise the question of the elementary units for which Zipf's law should hold in the most natural way,…
According to Zipf's meaning-frequency law, words that are more frequent tend to have more meanings. Here it is shown that a linear dependency between the frequency of a form and its number of meanings is found in a family of models of…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
Zipf's law has been found in many human-related fields, including language, where the frequency of a word is persistently found as a power law function of its frequency rank, known as Zipf's law. However, there is much dispute whether it is…