Related papers: Evolution and trade-off dynamics of functional loa…
While directly fine-tuning (FT) large-scale, pretrained models on task-specific data is well-known to induce strong in-distribution task performance, recent works have demonstrated that different adaptation protocols, such as linear probing…
When two (possibly different in distribution) continuous-state branching processes with immigration are present, we study the relative frequency of one of them when the total mass is forced to be constant at a dense set of times. This leads…
The relationship between genotype and phenotype plays a crucial role in determining the function and robustness of biological systems. Here the evolution progresses through the change in genotype, whereas the selection is based on the…
While language is a complex adaptive system, most work on syntactic variation observes a few individual constructions in isolation from the rest of the grammar. This means that the grammar, a network which connects thousands of structures…
Synonyms and homonyms appear in all natural languages. We analyse their evolution within the framework of the signaling game. Agents in our model use reinforcement learning, where probabilities of selection of a communicated word or of its…
Increasing evidence of the effects of changing climate on physical ocean conditions and long-term changes in fish populations adds to the need to understand the effects of stochastic forcing on marine populations. Cohort resonance is of…
Federated learning (FL) is a collaborative machine learning paradigm, which enables deep learning model training over a large volume of decentralized data residing in mobile devices without accessing clients' private data. Driven by the…
Fluctuation and dissipation dynamics is examined at all temperature ranges for the general case of a background time evolving scalar field coupled to heavy intermediate quantum fields which in turn are coupled to light quantum fields. The…
The class of tree-adjoining languages can be characterized by various two-level formalisms, consisting of a context-free grammar (CFG) or pushdown automaton (PDA) controlling another CFG or PDA. These four formalisms are equivalent to…
Discrete Diffusion Language Models progressed rapidly as an alternative to autoregressive (AR) models, motivated by their parallel generation abilities. However, for tractability, discrete diffusion models sample from a factorized…
Evolutionary dynamics are shaped by a variety of fundamental, generic drivers, including spatial structure, ecology, and selection pressure. These drivers impact the trajectory of evolution, and have been hypothesized to influence…
Value trade-offs are an integral part of human decision-making and language use, however, current tools for interpreting such dynamic and multi-faceted notions of values in language models are limited. In cognitive science, so-called…
In GFlowNets and variational inference, it has been shown that the mean square error between target and model log probabilities is an effective, low variance, surrogate loss for training generative models. This loss has the property that…
A condensate of pairs in an isolated, homogeneous, unpolarised, finite-size spin 1/2 Fermi gas at low nonzero temperature T, undergoes with time a phase change with a random component, due to coupling to the gas thermal phonons. With the…
Diffusion language models are a promising alternative to autoregressive models, yet post-training methods for them largely adapt reward-maximizing objectives. We identify a central failure mode in this setting we call trajectory locking:…
We consider a rotator whose equation of motion for the angle $\theta$ consists of the zeroth and first Fourier modes. Numerical analysis based on the trailing of saddle-node bifurcations is used to locate the n:1 Arnold tongues where…
The distribution of living languages is investigated and scaling relations are found for the diversity of languages as a function of the country area and population. These results are compared with data from Ecology and from computer…
Evolution is simultaneously driven by a number of processes such as mutation, competition and random sampling. Understanding which of these processes is dominating the collective evolutionary dynamics in dependence on system properties is a…
Despite their impressive capabilities, aligned large language models (LLMs) often generate outputs that lack diversity. What drives this consistency in the generation? We investigate this phenomenon through the lens of probability…
Since the advent of Federated Learning (FL), research has applied these methods to natural language processing (NLP) tasks. Despite a plethora of papers in FL for NLP, no previous works have studied how multilingual text impacts FL…