Related papers: Spin Glass Models of Syntax and Language Evolution
The phenomenon of human language is widely studied from various points of view. It is interesting not only for social scientists, antropologists or philosophers, but also for those, interesting in the network dynamics. In several recent…
The way the words are used evolves through time, mirroring cultural or technological evolution of society. Semantic change detection is the task of detecting and analysing word evolution in textual data, even in short periods of time. In…
In the past years, a remarkable mapping has been found between the dynamics of a population of M individuals undergoing random mutations and selection, and that of a single system in contact with a thermal bath with temperature 1/M. This…
The slow non-equilibrium dynamics of the Edwards-Anderson spin glass model on a hierarchical lattice is studied by means of a coarse-grained description based on renormalization concepts. We evaluate the isothermal aging properties and show…
Large neural language models are steadily contributing state-of-the-art performance to question answering and other natural language and information processing tasks. These models are expensive to train. We propose to evaluate whether such…
Languages continually evolve in response to societal events, resulting in new terms and shifts in meanings. These changes have significant implications for computer applications, including automatic translation and chatbots, making it…
Transformer language models have received widespread public attention, yet their generated text is often surprising even to NLP researchers. In this survey, we discuss over 250 recent studies of English language model behavior before…
Language is an intricately structured system, and a key goal of NLP interpretability is to provide methodological insights for understanding how language models represent this structure internally. In this paper, we use Shapley Taylor…
The syntactic structures of sentences can be readily read-out from the activations of large language models (LLMs). However, the ``structural probes'' that have been developed to reveal this phenomenon are typically evaluated on an…
We study evolutionary canalization using a spin-glass model with replica theory, where spins and their interactions are dynamic variables whose configurations correspond to phenotypes and genotypes, respectively. The spins are updated under…
Similarities between fragile glasses and spin glasses (SG) suggest the study of frustrated spin model to understand the complex dynamics of glasses above the glass transition. We consider a frustrated spin model with Ising spins and s-state…
Computer model has been extensively adopted to overcome the time limitation of language evolution by transforming language theory into physical modeling mechanism, which helps to explore the general laws of the evolution. In this paper, a…
In this paper, we review the general features of the out-of-equilibrium dynamics of spin glasses. We use this example as a guideline for a brief description of glassy dynamics in other disordered systems like structural and polymer glasses,…
With the advent of powerful neural language models over the last few years, research attention has increasingly focused on what aspects of language they represent that make them so successful. Several testing methodologies have been…
In the following article, we construct an interaction model (a variant of the SIR-model) of general language change. In the context of language change it is desirable to deduce the long-term behaviour of the corresponding dynamical system…
Models of spin glasses are studied with a phase transition discontinuous in the Parisi order parameter. It is assumed that the leading order corrections to the thermodynamic limit of the high temperature free energy are due to the existence…
A model is derived for the viscoelastic response of glasses at isothermal uniaxial deformation with small strains. A glass is treated as an ensemble of relaxing units with various activation energies for rearrangement. With reference to the…
Mathematical modeling of real-world physical systems requires the consistent combination of a multitude of physical laws and phenomenological models. This challenging task can be greatly simplified by hierarchically decomposing systems into…
We use Monte Carlo (MC) methods to simulate a two-dimensional (2D) bond-diluted Ising model on the square lattice which has frustration between the nearest-neighbor interaction J1 and the next-nearest-neighbor interaction J2. In this paper,…
We study energy landscape and dynamics of the three-dimensional Heisenberg Spin Glass model in the paramagnetic phase, i.e. for temperature $T$ larger than the critical temperature $T_\mathrm{c}$. The landscape is non-trivially related to…