Related papers: The Rank-Size Scaling Law and Entropy-Maximizing P…
We study the problem of entropy calibration, which asks whether a language model's entropy over generations matches its log loss on human text. Past work found that models are miscalibrated, with entropy per step increasing as generations…
Quantitative linguistics has provided us with a number of empirical laws that characterise the evolution of languages and competition amongst them. In terms of language usage, one of the most influential results is Zipf's law of word…
The principle of entropy increase is not only the basis of statistical mechanics, but also closely related to the irreversibility of time, the origin of life, chaos and turbulence. In this paper, we first discuss the dynamic system…
Metcalfe's Law captures the relationship between the value of a network and its scale, asserting that a network's value is directly proportional to the square of its size. Over the past four decades, various researchers have proposed…
Entropy estimation of information sources is highly non trivial for symbol sequences with strong long-range correlations. The rabbit sequence, related to the symbolic dynamics of the nonlinear circle map at the critical point as well as the…
Human language, as a typical complex system, its organization and evolution is an attractive topic for both physical and cultural researchers. In this paper, we present the first exhaustive analysis of the text organization of human speech.…
The derivation of the maximum entropy distribution of particles in boxes yields two kinds of distributions: a "bell-like" distribution and a long-tail distribution. The first one is obtained when the ratio between particles and boxes is…
The size and number of maximum matchings in a network have found a large variety of applications in many fields. As a ubiquitous property of diverse real systems, power-law degree distribution was shown to have a profound influence on size…
Using an exhaustive list of Japanese bankruptcy in 1997, we discover a Zipf law for the distribution of total liabilities of bankrupted firms in high debt range. The life-time of these bankrupted firms has exponential distribution in…
We demonstrate the emergence of scaling laws in the benchmark top versus QCD jet classification problem in collider physics. Six distinct physically-motivated classifiers exhibit power-law scaling of the binary cross-entropy test loss as a…
Research in network science has shown that many naturally occurring and technologically constructed networks are scale free, that means a power law degree distribution emerges from a growth model in which each new node attaches to the…
We show that statistical criticality, i.e. the occurrence of power law frequency distributions, arises in samples that are maximally informative about the underlying generating process. In order to reach this conclusion, we first identify…
In a language corpus, the probability that a word occurs $n$ times is often proportional to $1/n^2$. Assigning rank, $s$, to words according to their abundance, $\log s$ vs $\log n$ typically has a slope of minus one. That simple Zipf's law…
Scaling laws in ecology, intended both as functional relationships among ecologically-relevant quantities and the probability distributions that characterize their occurrence, have long attracted the interest of empiricists and…
We show that the natural scaling of measurement for a particular problem defines the most likely probability distribution of observations taken from that measurement scale. Our approach extends the method of maximum entropy to use…
Many complex systems are characterized by non-Boltzmann distribution functions of their statistical variables. If one wants to -- justified or not -- hold on to the maximum entropy principle for complex statistical systems (non-Boltzmann)…
Scaling and hyperscaling laws provide exact relations among critical exponents describing the behavior of a system at criticality. For nonequilibrium growth models with a conserved drift there exist few of them. One such relation is $\alpha…
We propose a method to derive the stationary size distributions of a system, and the degree distributions of networks, using maximisation of the Gibbs-Shannon entropy. We apply this to a preferential attachment-type algorithm for systems of…
In social networks, it is conventionally thought that two individuals with more overlapped friends tend to establish a new friendship, which could be stated as homophily breeding new connections. While the recent hypothesis of maximum…
An expression is proposed for determining the error caused on entropy estimates by finite sample effects. This expression is based on the Ansatz that the ranked distribution of probabilities tends to follow an empirical Zipf law.