Related papers: Neutral evolution: A null model for language dynam…
Humans continuously adapt their style and language to a variety of domains. However, a reliable definition of `domain' has eluded researchers thus far. Additionally, the notion of discrete domains stands in contrast to the multiplicity of…
We study a general setting of neutral evolution in which the population is of finite, constant size and can have spatial structure. Mutation leads to different genetic types ("traits"), which can be discrete or continuous. Under minimal…
We propose a stochastic model for evolution. Births and deaths of species occur with constant probabilities. Each new species is associated with a fitness sampled from the uniform distribution on [0,1]. Every time there is a death event…
This paper pursues the insight that language models naturally enable an intelligent variation operator similar in spirit to evolutionary crossover. In particular, language models of sufficient scale demonstrate in-context learning, i.e.…
Populations of competing biological species exhibit a fascinating interplay between the nonlinear dynamics of evolutionary selection forces and random fluctuations arising from the stochastic nature of the interactions. The processes…
The inheritance of characteristics induced by the environment has often been opposed to the theory of evolution by natural selection. Yet, while evolution by natural selection requires new heritable traits to be produced and transmitted, it…
Negation is a fundamental linguistic operator, yet it remains inadequately modeled in diffusion-based generative systems. In this work, we present a formal treatment of linguistic negation in diffusion-based generative models by modeling it…
Our understanding of the evolutionary process has gone a long way since the publication, 150 years ago, of "On the origin of species" by Charles R. Darwin. The XXth Century witnessed great efforts to embrace replication, mutation, and…
Competition is the main driver of population dynamics, which shapes the genetic composition of populations and the assembly of ecological communities. Neutral models assume that all the individuals are equivalent and that the dynamics is…
Many complex generative systems use languages to create structured objects. We consider a model of random languages, defined by weighted context-free grammars. As the distribution of grammar weights broadens, a transition is found from a…
Our knowledge of past cultures relies considerably on written material. For centuries, texts have been copied, altered, then transmitted or lost - eventually, from surviving documents, philologists attempt to reconstruct text phylogenies…
Influencing various aspects of human activity, migration is associated also with language formation. To examine the mutual interaction of these processes, we study a Naming Game with migrating agents. The dynamics of the model leads to…
Dynamically Adaptive Systems modify their behav- ior and structure in response to changes in their surrounding environment and according to an adaptation logic. Critical sys- tems increasingly incorporate dynamic adaptation capabilities;…
Neural conversation models are attractive because one can train a model directly on dialog examples with minimal labeling. With a small amount of data, however, they often fail to generalize over test data since they tend to capture…
As generative models become ubiquitous, there is a critical need for fine-grained control over the generation process. Yet, while controlled generation methods from prompting to fine-tuning proliferate, a fundamental question remains…
Recent studies in zero-shot cross-lingual learning using multilingual models have falsified the previous hypothesis that shared vocabulary and joint pre-training are the keys to cross-lingual generalization. Inspired by this advancement, we…
Natural language generation models reproduce and often amplify the biases present in their training data. Previous research explored using sequence-to-sequence rewriting models to transform biased model outputs (or original texts) into more…
Languages emerge and change over time at the population level though interactions between individual speakers. It is, however, hard to directly observe how a single speaker's linguistic innovation precipitates a population-wide change in…
Existing theoretical models of evolution focus on the relative fitness advantages of different mutants in a population while the dynamic behavior of the population size is mostly left unconsidered. We here present a generic stochastic model…
This chapter critically examines the potential contributions of modern language models to theoretical linguistics. Despite their focus on engineering goals, these models' ability to acquire sophisticated linguistic knowledge from mere…