Related papers: Spin Glass Models of Syntax and Language Evolution

We propose a model for the evolutionary ecology of words as one attempt to extend evolutionary game theory and agent-based models by utilizing the rich linguistic expressions of Large Language Models (LLMs). Our model enables the emergence…

We present a systematic review of 337 articles evaluating the syntactic abilities of Transformer-based language models (TLMs), reporting on over 3,000 datapoints spanning a wide range of syntactic phenomena, languages, models, and methods.…

In a $p$-spin interaction spherical spin-glass model both the spins and the couplings are allowed to change in the course of time. The spins are coupled to a heat bath with temperature $T$, while the coupling constants are coupled to a bath…

Though languages can evolve slowly, they can also react strongly to dramatic world events. By studying the connection between words and events, it is possible to identify which events change our vocabulary and in what way. In this work, we…

Despite impressive success, language models often generate outputs with flawed linguistic structure. We analyze the effect of directly infusing various kinds of syntactic and semantic information into large language models. To demonstrate…

This chapter critically examines the potential contributions of modern language models to theoretical linguistics. Despite their focus on engineering goals, these models' ability to acquire sophisticated linguistic knowledge from mere…

We consider the equilibrium dynamics of Ising spin models with multi-spin interactions on sparse random graphs (Bethe lattices). Such models undergo a mean field glass transition upon increasing the graph connectivity or lowering the…

We explore a one-to-one correspondence between a neural network (NN) and a statistical mechanical spin model where neurons are mapped to Ising spins and weights to spin-spin couplings. The process of training an NN produces a family of spin…

Neural language models (LMs) perform well on tasks that require sensitivity to syntactic structure. Drawing on the syntactic priming paradigm from psycholinguistics, we propose a novel technique to analyze the representations that enable…

The grammars of natural languages may be learned by using genetic algorithms that reproduce and mutate grammatical rules and part-of-speech tags, improving the quality of later generations of grammatical components. Syntactic rules are…

Computational modelling with multi-agent systems is becoming an important technique of studying language evolution. We present a brief introduction into this rapidly developing field, as well as our own contributions that include an…

Dilute magnetic nanoparticle systems exhibit slow dynamics [1] due to a broad distribution of relaxation times that can be traced to a correspondingly broad distribution of particle sizes [1]. However, at higher concentrations interparticle…

Syntactic theory has traditionally adopted a constructivist approach, in which a set of atomic elements are manipulated by combinatory operations to yield derived, complex elements. Syntactic structure is thus seen as the result or discrete…

We consider the Potts model on a two-dimensional periodic rectangular lattice with general coupling constants $J_{ij}>0$, where $i,j\in\{1,2,3\}$ are the possible spin values (or colors). The resulting energy landscape is thus significantly…

A zero temperature dynamics of Ising spin glasses and ferromagnets on random graphs of finite connectivity is considered, like granular media these systems have an extensive entropy of metastable states. We consider the problem of what…

Languages are shaped by the inductive biases of their users. Using a classical referential game, we investigate how artificial languages evolve when optimised for inductive biases in humans and large language models (LLMs) via Human-Human,…

This research aims to investigate the dynamic nature of linguistic style throughout various stages of life, from post teenage to old age. By employing linguistic analysis tools and methodologies, the study will delve into the intricacies of…

We analyze operational risk in terms of a spin glass model. Several regimes are investigated, as a functions of the parameters that characterize the dynamics. The system is found to be robust against variations of these parameters. We…

Word embeddings are a powerful approach for unsupervised analysis of language. Recently, Rudolph et al. (2016) developed exponential family embeddings, which cast word embeddings in a probabilistic framework. Here, we develop dynamic…

We use the persistent homology method of topological data analysis and dimensional analysis techniques to study data of syntactic structures of world languages. We analyze relations between syntactic parameters in terms of dimensionality,…