Related papers: Syntactic Structures and Code Parameters
Large language models often achieve strong benchmark gains without corresponding improvements in broader capability. We hypothesize that this discrepancy arises from differences in training regimes induced by data distribution. To…
In this note, a class of error-correcting codes is associated to a toric variety associated to a fan defined over a finite field $\fff_q$, analogous to the class of Goppa codes associated to a curve. For such a ``toric code'' satisfying…
We investigate the influence of static correlations beyond the pair level on the dynamics of selected model glass-formers. We compare the pair structure, angular distribution functions, and statistics of Voronoi polyhedra of two well-known…
This paper reviews some results regarding symbolic dynamics, correspondence between languages of dynamical systems and combinatorics. Sturmian sequences provide a pattern for investigation of one-dimensional systems, in particular interval…
Is it possible to develop a `physics of language' which can explain the spatial, temporal and social patterns we see, and which can predict future change like we forecast the weather? Such a theory is likely to involve ideas from…
Broken-symmetry-induced order parameters account for many phenomena in condensed matter physics. For spin glasses, such a framework dictates its theoretical construction, whereas experiments have only established dynamical behaviors such as…
We report on a refined version of our spin-glass type approach to the low-temperature physics of structural glasses. Its key idea is based on a Born von Karman expansion of the interaction potential about a set of reference positions in…
Latency reduction of postsynaptic spikes is a well-known effect of Synaptic Time-Dependent Plasticity. We expand this notion for long postsynaptic spike trains, showing that, for a fixed input spike train, STDP reduces the number of…
The recent dramatic increase in online data availability has allowed researchers to explore human culture with unprecedented detail, such as the growth and diversification of language. In particular, it provides statistical tools to explore…
Glass transition where viscosity of liquids increases dramatically upon decrease of temperature without any major change in structural properties, remains one of the most challenging problems in condensed matter physics (Cavagna, 2009;…
Although information theoretic characterizations of human communication have become increasingly popular in linguistics, to date they have largely involved grafting probabilistic constructs onto older ideas about grammar. Similarities…
Spatially extended population dynamics models that incorporate intrinsic noise serve as case studies for the role of fluctuations and correlations in biological systems. Including spatial structure and stochastic noise in predator-prey…
We investigate mechanisms for language change within a framework where an unconventional signal for a meaning is first innovated, and then subsequently propagated through a speech community to replace the existing convention. We appeal to…
Methods and insights from statistical physics are finding an increasing variety of applications where one seeks to understand the emergent properties of a complex interacting system. One such area concerns the dynamics of language at a…
The statistical physics properties of low-density parity-check codes for the binary symmetric channel are investigated as a spin glass problem with multi-spin interactions and quenched random fields by the cavity method. By evaluating the…
A simple geometrical characterization of configuration space neighborhoods of local energy minima in spin glass landscapes is found by exhaustive search. Combined with previous Monte Carlo investigations of thermal domain growth, it allows…
Weak superimposed codes are combinatorial structures related closely to generalized cover-free families, superimposed codes, and disjunct matrices in that they are only required to satisfy similar but less stringent conditions. This class…
The embedding space of language models is widely believed to capture the semantic relationships; for instance, embeddings of digits often exhibit an ordered structure that corresponds to their natural sequence. However, the mechanisms…
The topological nature of the disorder of glasses and supercooled liquids strongly affects their high-frequency dynamics. In order to understand its main features, we analytically studied a simple topologically disordered model, where the…
We consider unsupervised learning methods for characterizing the disordered microscopic structure of supercooled liquids and glasses. Specifically, we perform dimensionality reduction of smooth structural descriptors that describe radial…