Related papers: Syntactic Structures and Code Parameters
Analytic combinatorics in several variables refers to a suite of tools that provide sharp asymptotic estimates for certain combinatorial quantities. In this paper, we apply these tools to determine the Gilbert--Varshamov lower bound on the…
Statistical regularities in human language have fascinated researchers for decades, suggesting deep underlying principles governing its evolution and information structuring for efficient communication. While Zipf's Law describes the…
Syntax is a latent hierarchical structure which underpins the robust and compositional nature of human language. In this work, we explore the hypothesis that syntactic dependencies can be represented in language model attention…
We describe our perspective on the Structural Glass Transition (SGT) problem built on the premise that a viable theory must provide a consistent picture of the dynamics and statics, which are manifested by large increase in shear viscosity…
This paper analyses the degree to which dialect classifiers based on syntactic representations remain stable over space and time. While previous work has shown that the combination of grammar induction and geospatial text classification…
The correlation between local structure and the propensity for structural rearrangements has been widely investigated in glass forming liquids and glasses. In this paper we use the excess two-body entropy $S_2$ and tetrahedrality $\n_{tet}$…
Within the mode-coupling theory for idealized glass transitions, we study the evolution of structural relaxation in binary mixtures of hard spheres with size ratios $\delta$ of the two components varying between 0.5 and 1.0. We find two…
Targeted syntactic evaluations of language models ask whether models show stable preferences for syntactically acceptable content over minimal-pair unacceptable inputs. Most targeted syntactic evaluation datasets ask models to make these…
Probabilistic embeddings have several advantages over deterministic embeddings as they map each data point to a distribution, which better describes the uncertainty and complexity of data. Many works focus on adjusting the distribution…
We investigate the possibility of modelling the syntax and semantics of natural language by constraints, or rules, imposed by the multi-dimensional type theory Nabla. The only multiplicity we explicitly consider is two, namely one dimension…
Syntactic language models (SLMs) enhance Transformers by incorporating syntactic biases through the modeling of linearized syntactic parse trees alongside surface sentences. This paper focuses on compositional SLMs that are based on…
We summarize a theoretical framework based on global time-reparametrization invariance that explains the origin of dynamic fluctuations in glassy systems. We introduce the main ideas without getting into much technical details. We describe…
Language sciences rely less and less on formal syntax as their base. The reason is probably its lack of psychological reality, knowingly avoided. Philosophers of science call for a paradigm shift in which explanations are by mechanisms, as…
We investigate the asymptotic density of error-correcting codes with good distance properties and prescribed linearity degree, including sublinear and nonlinear codes. We focus on the general setting of finite translation-invariant metric…
We consider an infinite system of coupled stochastic differential equations (SDE) describing dynamics of the following infinite particle system. Each partricle is characterised by its position $x\in \mathbb{R}^{d}$ and internal parameter…
Traditionally, semantic models of imperative languages use an auxiliary structure which mimics memory. In this way, ownership and other encapsulation properties need to be reconstructed from the graph structure of such global memory. We…
While language models demonstrate sophisticated syntactic capabilities, the extent to which their internal mechanisms align with cross-constructional principles studied in linguistics remains poorly understood. This study investigates…
A central question in psycholinguistics is the timing of syntax in sentence processing. Much of the existing evidence comes from violation paradigms, which conflate two separable processes - syntactic category detection and phrase structure…
By characterizing the dynamics of idealized lattice models with a tunable kinetic constraint, we explore the different ways in which dynamical facilitation manifests itself within the local dynamics of glassy materials. Dynamical…
Controlling the syntactic structure of text generated by language models is valuable for applications requiring clarity, stylistic consistency, or interpretability, yet it remains a challenging task. In this paper, we argue that sampling…