Related papers: Spelling Rules for the Monster/Semple Tower
The possibility that mirror matter with masses in the several hundred GeV- TeV range exists is explored. Mirror matter appears quite naturally in many unified models of particle interactions both in GUTs and in strings often in vector-like…
Language tasks involving character-level manipulations (e.g., spelling corrections, arithmetic operations, word games) are challenging for models operating on subword units. To address this, we develop a causal intervention framework to…
We discuss a notion of convergence for binary trees that is based on subtree sizes. In analogy to recent developments in the theory of graphs, posets and permutations we investigate some general aspects of the topology, such as a…
Complex machine learning algorithms are used more and more often in critical tasks involving text data, leading to the development of interpretability methods. Among local methods, two families have emerged: those computing importance…
In this paper, we build a reinforcement learning framework to study how children compose numbers using base-ten blocks. Studying numerical cognition in toddlers offers a powerful window into the learning process itself, because numbers sit…
We count the number of distinct (scattered) subwords occurring in the base-b expansion of the non-negative integers. More precisely, we consider the sequence $(S_b(n))_{n\ge 0}$ counting the number of positive entries on each row of a…
Construction grammar posits that language learners acquire constructions (form-meaning pairings) from the statistics of their environment. Recent work supports this hypothesis by showing sensitivity to constructions in pretrained language…
Modern language models mostly take sub-words as input, a design that balances the trade-off between vocabulary size, number of parameters, and performance. However, sub-word tokenization still has disadvantages like not being robust to…
Statements involving metalinguistic self-reference ("This paper has six sections.") are prevalent in many domains. Can current large language models (LLMs) handle such language? In this paper, we present "I am a Strange Dataset", a new…
We obtain error terms on the rate of convergence to Extreme Value Laws for a general class of weakly dependent stochastic processes. The dependence of the error terms on the `time' and `length' scales is very explicit. Specialising to data…
We propose Sentence Level Recurrent Topic Model (SLRTM), a new topic model that assumes the generation of each word within a sentence to depend on both the topic of the sentence and the whole history of its preceding words in the sentence.…
Given a positive integer $n$ and a partitioning $n=r_1s_1+\dots+ r_ts_t$, $t,r_i,s_i$ positive integers, such that $r_1>\dots>r_t$ (for $t\ge 2$), we can write $n$ symbols $1,\dots,n$ in the form of a staircase matrix having $r_1$ rows…
In this paper we study the problem of constructing non-trivial subtowers and supertowers of recursive towers of function fields over finite fields.
The rapid escalation from elementary school-level to frontier problems of the difficulty for LLM benchmarks in recent years have weaved a miracle for researchers that we are only inches away from surpassing human intelligence. However, is…
To determine whether an $n\times n$-matrix has rank at most $r$ it suffices to check that the $(r+1)\times (r+1)$-minors have rank at most $r$. In other words, to describe the set of $n\times n$-matrices with the property of having rank at…
This paper investigates the learning of 3rd-order tensors representing the semantics of transitive verbs. The meaning representations are part of a type-driven tensor-based semantic framework, from the newly emerging field of compositional…
An important question concerning contextualized word embedding (CWE) models like BERT is how well they can represent different word senses, especially those in the long tail of uncommon senses. Rather than build a WSD system as in previous…
While important properties of word vector representations have been studied extensively, far less is known about the properties of sentence vector representations. Word vectors are often evaluated by assessing to what degree they exhibit…
The word inference problem is to determine languages such that the information on the number of occurrences of those subwords in the language can uniquely identify a word. A considerable amount of work has been done on this problem, but the…
Modern language models like GPT-3, BERT, and LLaMA require massive training data, yet with sufficient training they reliably learn to distinguish grammatical from ungrammatical sentences. Children aged as young as 14 months already have the…