Related papers: Dynamics of text generation with realistic Zipf di…
Many studies in Economics and other disciplines have been reporting distributions following power-law behavior (i.e distributions of incomes (Pareto's law), city sizes (Zipf's law), frequencies of words in long sequences of text etc.)[1, 6,…
Why does Zipf's law give a good description of data from seemingly completely unrelated phenomena? Here it is argued that the reason is that they can all be described as outcomes of a ubiquitous random group division: the elements can be…
In this paper we deal with the structural properties of weighted networks. Starting from an empirical analysis of a linguistic network, we analyse the differences between the statistical properties of a real and a shuffled network and we…
Zipf-like distributions characterize a wide set of phenomena in physics, biology, economics and social sciences. In human activities, Zipf-laws describe for example the frequency of words appearance in a text or the purchases types in…
The dependence of the frequency distributions due to multiple meanings of words in a text is investigated by deleting letters. By coding the words with fewer letters the number of meanings per coded word increases. This increase is measured…
Data of proportional elections show a striking feature: If the parties are ranked according to the number of their voters, the number of votes grows exponentially with the rank of the party. This so-called Zipf's law has been reported…
The importance of statistical patterns of language has been debated over decades. Although Zipf's law is perhaps the most popular case, recently, Menzerath's law has begun to be involved. Menzerath's law manifests in language, music and…
n-tuple power law widely exists in language, computer program code, DNA and music. After a vast amount of Zipf analyses of n-tuple power law from empirical data, we propose a model to explain the n-tuple power law feature existed in these…
Long Short-Term Memory (LSTM) networks have recently shown remarkable performance in several tasks dealing with natural language generation, such as image captioning or poetry composition. Yet, only few works have analyzed text generated by…
Languages and genes are both transmitted from generation to generation, with opportunity for differential reproduction and survivorship of forms. Here we apply a rigorous inference framework, drawn from population genetics, to distinguish…
We summarize a book under publication with his title written by the three present authors, on the theory of Zipf's law, and more generally of power laws, driven by the mechanism of proportional growth. The preprint is available upon request…
It is generally believed that, when a linguistic item acquires a new meaning, its overall frequency of use in the language rises with time with an S-shaped growth curve. Yet, this claim has only been supported by a limited number of case…
This work studies discrete diffusion probabilistic models with applications to natural language generation. We derive an alternative yet equivalent formulation of the sampling from discrete diffusion processes and leverage this insight to…
Large language models generate text through probabilistic sampling from high-dimensional distributions, yet how this process reshapes the structural statistical organization of language remains incompletely characterized. Here we show that…
This paper presents the results of the application of a bit-string model of languages (Schulze and Stauffer 2005) to problems of taxonomic patterns. The questions addressed include the following: (1) Which parameters are minimally ne eded…
We introduce a simple and generic model that reproduces Zipf's law. By regarding the time evolution of the model as a random walk in the logarithmic scale, we explain theoretically why this model reproduces Zipf's law. The explanation shows…
A theory of systems with long-range correlations based on the consideration of binary N-step Markov chains is developed. In the model, the conditional probability that the i-th symbol in the chain equals zero (or unity) is a linear function…
This study investigates the consequences of training language models on synthetic data generated by their predecessors, an increasingly prevalent practice given the prominence of powerful generative models. Diverging from the usual emphasis…
It is shown that the distribution of low variability periods in the activity of human heart rate typically follows a multi-scaling Zipf's law. The presence or failure of a power law, as well as the values of the scaling exponents, are…
We present a comparison of word-based and character-based sequence-to-sequence models for data-to-text natural language generation, which generate natural language descriptions for structured inputs. On the datasets of two recent generation…