Related papers: Spin Glass Models of Syntax and Language Evolution
Pre-trained transformer language models (TLMs) have recently refashioned natural language processing (NLP): Most state-of-the-art NLP models now operate on top of TLMs to benefit from contextualization and knowledge induction. To explain…
This paper integrates graph-to-sequence into an end-to-end text-to-speech framework for syntax-aware modelling with syntactic information of input text. Specifically, the input text is parsed by a dependency parsing module to form a…
In recent times, the research field of language dynamics has focused on the investigation of language evolution, dividing the work in three evolutive steps, according to the level of complexity: lexicon, categories and grammar. The Naming…
We introduce a diluted version of the one dimensional spin-glass model with interactions decaying in probability as an inverse power of the distance. In this model varying the power corresponds to change the dimension in short-range models.…
The generalized Lotka-Volterra stochastic differential equation with a symmetric food interaction matrix is frequently used to model the dynamics of the abundances of the species living within an ecosystem when these interactions are…
Language models have been shown to be very effective in predicting brain recordings of subjects experiencing complex language stimuli. For a deeper understanding of this alignment, it is important to understand the correspondence between…
We develop a system that formally represents spatial semantics concepts within natural language descriptions of spatial arrangements. The system builds on a model of spatial semantics representation according to which words in a sentence…
During the last decade, much attention has been paid to language competition in the complex systems community, that is, how the fractions of speakers of several competing languages evolve in time. In this paper we review recent advances in…
This paper reports numerical studies of a compressible version of the Ising spin glass in two dimensions. Compressibility is introduced by adding a term that couples the spin-spin interactions and local lattice deformations to the standard…
We revisit the phenomenon of syntactic complexity convergence in conversational interaction, originally found for English dialogue, which has theoretical implication for dialogical concepts such as mutual understanding. We use a modified…
We study the problem of leveraging the syntactic structure of text to enhance pre-trained models such as BERT and RoBERTa. Existing methods utilize syntax of text either in the pre-training stage or in the fine-tuning stage, so that they…
Language Generation Models produce words based on the previous context. Although existing methods offer input attributions as explanations for a model's prediction, it is still unclear how prior words affect the model's decision throughout…
Spin glasses occupy a unique place in condensed matter: they freeze collectively while remaining struc-turally disordered, and they exhibit slow, history-dependent dynamics that reflect an exceptionally rug-ged free-energy landscape. This…
We study the dynamical low temperature behaviour of the Ising spin glass on the Bethe lattice. Starting from Glauber dynamics we propose a cavity like Ansatz that allows for the treatment of the slow (low temperature) part of dynamics.…
Spin Glasses (SG) are paradigmatic models for physical, computer science, biological and social systems. The problem of studying the dynamics for SG models is NP hard, i.e., no algorithm solves it in polynomial time. Here we implement the…
We study the $\pm J$ three-dimensional Ising model with a longitudinal anisotropic bond randomness on the simple cubic lattice. The random exchange interaction is applied only in the $z$ direction, whereas in the other two directions, $xy$…
Semantics, morphology and syntax are strongly interdependent. However, the majority of computational methods for semantic change detection use distributional word representations which encode mostly semantics. We investigate an alternative…
Kinetically constrained spin models are known to exhibit dynamical behavior mimicking that of glass forming systems. They are often understood as coarse-grained models of glass formers, in terms of some "mobility" field. The identity of…
Recent NLP literature has seen growing interest in improving model interpretability. Along this direction, we propose a trainable neural network layer that learns a global interaction graph between words and then selects more informative…
Large Language Models (LLMs) demonstrate strong mathematical problem-solving abilities but frequently fail on problems that deviate syntactically from their training distribution. We identify a systematic failure mode, syntactic blind…