Related papers: Multi-dimensional Boltzmann Sampling of Languages
We investigate the problem of percolation of words in a random environment. To each vertex, we independently assign a letter $0$ or $1$ according to Bernoulli r.v.'s with parameter $p$. The environment is the resulting graph obtained from…
The syntactic structure of a sentence is often represented using syntactic dependency trees. The sum of the distances between syntactically related words has been in the limelight for the past decades. Research on dependency distances led…
We consider the problem of computing the Maximal Exact Matches (MEMs) of a given pattern $P[1 .. m]$ on a large repetitive text collection $T[1 .. n]$, which is represented as a (hopefully much smaller) run-length context-free grammar of…
Generating natural language under complex constraints is a principled formulation towards controllable text generation. We present a framework to allow specification of combinatorial constraints for sentence generation. We propose TSMH, an…
Twins in a finite word are formed by a pair of identical subwords placed at disjoint sets of positions. We investigate the maximum length of twins in a random word over a $k$-letter alphabet. The obtained lower bounds for small values of…
We investigate the variance of the length of the longest common subsequences of two independent random words of size $n$, where the letters of one word are i.i.d. uniformly drawn from $\{\alpha_1, \alpha_2, \cdots, \alpha_m\}$, while the…
Today's probabilistic language generators fall short when it comes to producing coherent and fluent text despite the fact that the underlying models perform well under standard metrics, e.g., perplexity. This discrepancy has puzzled the…
In NLP, text language models based on words or subwords are known to outperform their character-based counterparts. Yet, in the speech community, the standard input of spoken LMs are 20ms or 40ms-long discrete units (shorter than a…
We present a famework for the automatic compilation of multi-parametric Boltzmann samplers for algebraic data types in Haskell. Our framework uses Template Haskell to synthesise efficient, entropy-optimal samplers generating random…
The d-neighborhood of a word W in the Levenshtein distance is the set of all words at distance at most d from W. Generating the neighborhood of a word W, or related sets of words such as the condensed neighborhood or the super-condensed…
Large Language Models achieve next-token prediction by transporting a vectorized piece of text (prompt) across an accompanying embedding space under the action of successive transformer layers. The resulting high-dimensional trajectories…
We survey several methods of generating large random lambda-terms, focusing on their closed and simply-typed variants. We discuss methods of exact- and approximate-size generation, as well as methods of achieving size-uniform and…
We enumerate all ternary length-l square-free words, which are words avoiding squares of words up to length l, for l<=24. We analyse the singular behaviour of the corresponding generating functions. This leads to new upper entropy bounds…
Word embedding models offer continuous vector representations that can capture rich contextual semantics based on their word co-occurrence patterns. While these word vectors can provide very effective features used in many NLP tasks such as…
Given a random text over a finite alphabet, we study the frequencies at which fixed-length words occur as subsequences. As the data size grows, the joint distribution of word counts exhibits a rich asymptotic structure. We investigate all…
Let $\widetilde{\alpha}$ be a length-$L$ cyclic sequence of characters from a size-$K$ alphabet $\mathcal{A}$ such that the number of occurrences of any length-$m$ string on $\mathcal{A}$ as a substring of $\widetilde{\alpha}$ is $\lfloor L…
Recent works on word representations mostly rely on predictive models. Distributed word representations (aka word embeddings) are trained to optimally predict the contexts in which the corresponding words tend to appear. Such models have…
We investigate the large deviations of the shape of the random RSK Young diagrams associated with a random word of size $n$ whose letters are independently drawn from an alphabet of size $m=m(n)$. When the letters are drawn uniformly and…
The success of large language models (LLMs) has motivated formal theories of language generation and learning. We study the framework of \emph{language generation in the limit}, where an adversary enumerates strings from an unknown language…
Some consider large-scale language models that can generate long and coherent pieces of text as dangerous, since they may be used in misinformation campaigns. Here we formulate large-scale language model output detection as a hypothesis…