Related papers: Utterance Selection Model of Language Change
We study the evolution of an oscillator interacting via the most general bilinear coupling (with time-independent coefficients) with an ``environment'' consisting of a set of other harmonic oscillators. We are mainly interested in a…
We adopt an evolutionary view on language change in which cognitive factors (in addition to social ones) affect the fitness of words and their success in the linguistic ecosystem. Specifically, we propose a variety of psycholinguistic…
The paper presents a language model that develops syntactic structure and uses it to extract meaningful information from the word history, thus enabling the use of long distance dependencies. The model assigns probability to every joint…
We study the role of the noise in the dynamics of two competing species. We consider generalized Lotka-Volterra equations in the presence of a multiplicative noise, which models the interaction between the species and the environment. The…
Building upon kinetic theory approaches for multi-agent systems and generalising them to scenarios where the total mass of the system is not conserved, we develop a modelling framework for phenotype-structured populations that makes it…
We show that across architecture (Transformer vs. Mamba vs. RWKV), training dataset (OpenWebText vs. The Pile), and scale (14 million parameters to 12 billion parameters), autoregressive language models exhibit highly consistent patterns of…
One of the most striking effect of fluctuations in evolutionary game theory is the possibility for mutants to fixate (take over) an entire population. Here, we generalize a recent WKB-based theory to study fixation in evolutionary games…
We introduce time variation in the flip-rates of the Voter Model. This type of generalisation is relevant to models of ageing in language change, allowing the representation of changes in speakers' learning rates over their lifetime and may…
Multispecies ecosystems modelled by generalized Lotka-Volterra equations exhibit stationary population abundances, where large number of species often coexist. Understanding the precise conditions under which this is at all feasible and…
The replicator-mutator dynamic was originally derived to model the evolution of language, and since the model was derived in such a general manner, it has been applied to the dynamics of social behavior and decision making in multi-agent…
The paper presents a language model that develops syntactic structure and uses it to extract meaningful information from the word history, thus enabling the use of long distance dependencies. The model assigns probability to every joint…
An equation describing the evolution of phenotypic distribution is derived using methods developed in statistical physics. The equation is solved by using the singular perturbation method, and assuming that the number of bases in the…
The ever-growing size of the foundation language model has brought significant performance gains in various types of downstream tasks. With the existence of side-effects brought about by the large size of the foundation language model such…
We study the relaxation to equilibrium for a class linear one-dimensional Fokker-Planck equations characterized by a particular subcritical confinement potential. An interesting feature of this class of Fokker-Planck equations is that, for…
Questions related to the evolution of language have recently known an impressive increase of interest (Briscoe, 2002). This short paper aims at questioning the scientific status of these models and their relations to attested data. We show…
Understanding how stochastic and non-linear deterministic processes interact is a major challenge in population dynamics theory. After a short review, we introduce a stochastic individual-centered particle model to describe the evolution in…
We address the well-posedness of subelliptic Fokker-Planck equations arising from stochastic control problems, as well as the properties of the associated diffusion processes. Here, the main difficulty arises from the possible polynomial…
We study the ABC model (A + B --> 2B, B + C --> 2C, C + A --> 2A), and its counterpart: the three--component neutral drift model (A + B --> 2A or 2B, B + C --> 2B or 2C, C + A --> 2C or 2A.) In the former case, the mean field approximation…
We analyze a general theory for coexistence and extinction of ecological communities that are influenced by stochastic temporal environmental fluctuations. The results apply to discrete time (stochastic difference equations), continuous…
In a physical system, changing parameters such as temperature can induce a phase transition: an abrupt change from one state of matter to another. Analogous phenomena have recently been observed in large language models. Typically, the task…