Related papers: Syntactic Structures and Code Parameters
We examine learning dynamics in deep recurrent networks, focusing on the behavior near the boundary in the depth-width plane separating under- from over-parametrized networks, known as the interpolation transition. The training data are…
A covering code is a set of codewords with the property that the union of balls, suitably defined, around these codewords covers an entire space. Generally, the goal is to find the covering code with the minimum size codebook. While most…
We propose transfer learning as a method for analyzing the encoding of grammatical structure in neural language models. We train LSTMs on non-linguistic data and evaluate their performance on natural language to assess which kinds of data…
Spatially coupled codes have been shown to universally achieve the capacity for a large class of channels. Many variants of such codes have been introduced to date. We discuss a further such variant that is particularly simple and is…
In this work, we focus on intrasentential code-mixing and propose several different Synthetic Code-Mixing (SCM) data augmentation methods that outperform the baseline on downstream sentiment analysis tasks across various amounts of labeled…
We compute the code parameters for binary linear codes obtained by greedy constructing the parity check matrix. Then we show that these codes improve the Gilbert-Varshamov (GV) bound on the code size and rate. This result counter proves the…
We study the glass transition by exploring a broad class of kinetic rules that can significantly modify the normal dynamics of super-cooled liquids, while maintaining thermal equilibrium. Beyond the usual dynamics of liquids, this class…
Phoneme frequency distributions exhibit robust statistical regularities across languages, including exponential-tailed rank-frequency patterns and a negative relationship between phonemic inventory size and the relative entropy of the…
We study cooperating distributed grammar systems working in hybrid modes in connection with the finite index restriction in two different ways: firstly, we investigate cooperating distributed grammar systems working in hybrid modes which…
Syntax connects words to each other in very specific ways. Two words are syntactically connected if they depend directly on each other. Syntactic connections usually happen within a sentence. Gathering all those connection across several…
Dynamics induced by a change of boundary conditions reveals rate-dependent signatures associated with topological properties in one-dimensional Kitaev chain and SSH model. While the perturbation from a change of the boundary propagates into…
We present a theory for the dynamics of a binary mixture with particle size swaps. The theory is based on a factorization approximation similar to that employed in the mode-coupling theory of glassy dynamics. The theory shows that, in…
Spin glasses have competing interactions that lead to a rough energy landscape which is highly susceptible to small perturbations. These chaotic effects strongly affect numerical simulations and, as such, gaining a deeper understanding of…
Source code processing heavily relies on the methods widely used in natural language processing (NLP), but involves specifics that need to be taken into account to achieve higher quality. An example of this specificity is that the semantics…
Complex networks theory has commonly been used for modelling and understanding the interactions taking place between the elements composing complex systems. More recently, the use of generative models has gained momentum, as they allow…
A growing body of theoretical and empirical evidence shows that the global steady-state distributions of many equilibrium and nonequilibrium systems approximately satisfy an analogue of the Boltzmann distribution, with a local dynamical…
Targeted syntactic evaluations have demonstrated the ability of language models to perform subject-verb agreement given difficult contexts. To elucidate the mechanisms by which the models accomplish this behavior, this study applies causal…
Why do bilingual speakers code-switch (mix their two languages)? Among the several theories that attempt to explain this natural and ubiquitous phenomenon, the Triggering Hypothesis relates code-switching to the presence of lexical…
We discuss the general link between mode-coupling like equations (which serve as the basis of some recent theories of supercooled liquids) and the dynamical equations governing mean-field spin-glass models, or the dynamics of a particle in…
We consider the classical coupled, combined-field integral equation formulations for time-harmonic acoustic scattering by a sound soft bounded obstacle. In recent work, we have proved lower and upper bounds on the $L^2$ condition numbers…