Related papers: Simple stochastic processes behind Menzerath's Law
This brief pedagogical note re-proves a simple theorem on the convergence, in $L_2$ and in probability, of time averages of non-stationary time series to the mean of expectation values. The basic condition is that the sum of covariances…
Construction grammar posits that constructions, or form-meaning pairings, are acquired through experience with language (the distributional learning hypothesis). But how much information about constructions does this distribution actually…
We present a theoretical and empirical investigation of the statistical behaviour of the words in a text produced by human language. To this aim, we analyse the word distribution of various texts of Italian language selected from a specific…
A random variable (r.v.) X is said to follow Benford's law if log(X) is uniform mod 1. Many experimental data sets prove to follow an approximate version of it, and so do many mathematical series and continuous random variables. This…
The classical fluctuation-dissipation theorem predicts the average response of a dynamical system to an external deterministic perturbation via time-lagged statistical correlation functions of the corresponding unperturbed system. In this…
Language models must capture statistical dependencies between words at timescales ranging from very short to very long. Earlier work has demonstrated that dependencies in natural language tend to decay with distance between words according…
The projected normal distribution, also known as the angular Gaussian distribution, is obtained by dividing a multivariate normal random variable $\mathbf{x}$ by its norm $\sqrt{\mathbf{x}^T \mathbf{x}}$. The resulting random variable…
We replicate a renewal process at random times, which is equivalent to nesting two renewal processes, or considering a renewal process subject to stochastic resetting. We investigate the consequences on the statistical properties of the…
We study the asymptotic behavior of stochastic hyperbolic parabolic equations with slow and fast time scales. Both the strong and weak convergence in the averaging principe are established, which can be viewed as a functional law of large…
We describe a novel algorithm for random sampling of freely reduced words equal to the identity in a finitely presented group. The algorithm is based on Metropolis Monte Carlo sampling. The algorithm samples from a stretched Boltzmann…
Time evolution of the cities and of the languages is considered in terms of multiplicative noise and fragmentation processes; where power law (Pareto-Zipf law) and slightly asymmetric log-normal (Gauss) distribution result for the size…
Mathematical models of motility are often based on random-walk descriptions of discrete individuals that can move according to certain rules. It is usually the case that large masses concentrated in small regions of space have a great…
We present a simple structure based model of how words are formed from morphemes. The model explains two major empirical facts: the typical distribution of word lengths and the appearance of Zipf like rank frequency curves. In contrast to…
Using a model based on generalised Lotka Volterra dynamics together with some recent results for the solution of generalised Langevin equations, we show that the equilibrium solution for the probability distribution of wealth has two…
For many standard models of random structure, first-order logic sentences exhibit a convergence phenomenon on random inputs. The most well-known example is for random graphs with constant edge probability, where the probabilities of…
In this paper, we study the ability of large language models to learn specific mathematical rules such as distributivity or simplifying equations. We present an empirical analysis of their ability to generalize these rules, as well as to…
In this paper we consider the normalized lengths of the factors of some factorizations of random words. First, for the \emph{Lyndon factorization} of finite random words with $n$ independent letters drawn from a finite or infinite totally…
Zipf's law of abbreviation, namely the tendency of more frequent words to be shorter, has been viewed as a manifestation of compression, i.e. the minimization of the length of forms -- a universal principle of natural communication.…
The grammars of natural languages may be learned by using genetic algorithms that reproduce and mutate grammatical rules and part-of-speech tags, improving the quality of later generations of grammatical components. Syntactic rules are…
Here we sketch a new derivation of Zipf's law for word frequencies based on optimal coding. The structure of the derivation is reminiscent of Mandelbrot's random typing model but it has multiple advantages over random typing: (1) it starts…