Related papers: Spin Glass Models of Syntax and Language Evolution
The evolution of natural languages poses a riddle to any theoretical perspective based on efficiency considerations. If languages are already optimally effective means of organization and communication of thought, why do they change? And if…
We assign binary and ternary error-correcting codes to the data of syntactic structures of world languages and we study the distribution of code points in the space of code parameters. We show that, while most codes populate the lower…
Large language models show a surprising in-context learning ability -- being able to use a prompt to form a prediction for a query, yet without additional training, in stark contrast to old-fashioned supervised learning. Providing a…
In the last half-decade, the field of natural language processing (NLP) has undergone two major transitions: the switch to neural networks as the primary modeling paradigm and the homogenization of the training regime (pre-train, then…
We cast the metabolism of interacting cells within a statistical mechanics framework considering both, the actual phenotypic capacities of each cell and its interaction with its neighbors. Reaction fluxes will be the components of…
The thesis presents an attempt at using the syntactic structure in natural language for improved language models for speech recognition. The structured language model merges techniques in automatic parsing and language modeling using an…
Nearest neighbors in word embedding models are commonly observed to be semantically similar, but the relations between them can vary greatly. We investigate the extent to which word embedding models preserve syntactic interchangeability, as…
Language sciences rely less and less on formal syntax as their base. The reason is probably its lack of psychological reality, knowingly avoided. Philosophers of science call for a paradigm shift in which explanations are by mechanisms, as…
The paper presents a language model that develops syntactic structure and uses it to extract meaningful information from the word history, thus enabling the use of long distance dependencies. The model assigns probability to every joint…
In this work we study spin-glass (SG) like behavior in the dynamics of multiple agents in a social or economic context using interactions which are similar to the physical case. The different preferences shown by individual agents are…
Accurate syntactic representations are essential for robust generalization in natural language. Recent work has found that pre-training can teach language models to rely on hierarchical syntactic features - as opposed to incorrect linear…
The evolution of high-dimensional phenotypes is investigated using a statistical physics model consists of interacting spins, in which genotypes, phenotypes, and environments are represented by spin configurations, interaction matrices, and…
Spatial linguistic surveys often reveal well defined geographical zones where certain linguistic forms are dominant over their alternatives. It has been suggested that these patterns may be understood by analogy with coarsening in models of…
Contact between languages has the potential to transmit vocabulary and other language features; however, this does not always happen. Here, an iterated learning model is used to examine, in a simple way, the resistance of languages to…
The evolution of grammatical systems of syntactic and semantic composition is modeled here with a novel application of reinforcement learning theory. To test the functionalist thesis that speakers' expressive purposes shape their language,…
A spin glass is a diluted magnetic material in which the magnetic moments are randomly interacting, with a huge number of metastable states which prevent reaching equilibrium. Spin-glass models are conceptually simple, but require very…
Spin glasses are paradigmatic models that deliver concepts relevant for a variety of systems. However, rigorous analytical results are difficult to obtain for spin-glass models, in particular for realistic short-range models. Therefore…
Language models (LMs) are capable of acquiring elements of human-like syntactic knowledge. Targeted syntactic evaluation tests have been employed to measure how well they form generalizations about syntactic phenomena in high-resource…
A natural next step in the evolution of constraint-based grammar formalisms from rewriting formalisms is to abstract fully away from the details of the grammar mechanism---to express syntactic theories purely in terms of the properties of…
Language models (LMs) trained on large quantities of text have been claimed to acquire abstract linguistic representations. Our work tests the robustness of these abstractions by focusing on the ability of LMs to learn interactions between…