Related papers: Packing patterns into words
Human languages vary widely in how they encode information within circumscribed semantic domains (e.g., time, space, color, human body parts and activities), but little is known about the global structure of semantic information and nothing…
We introduce an algorithm for the uniform generation of infinite traces, i.e., infinite words up to commutation of some letters. The algorithm outputs on-the-fly approximations of a theoretical infinite trace, the latter being distributed…
An infinite permutation is a linear order on the set N. We study the properties of infinite permutations generated by fixed points of some uniform binary morphisms, and find the formula for their complexity.
Permutation entropy measures the complexity of deterministic time series via a data symbolic quantization consisting of rank vectors called ordinal patterns or just permutations. The reasons for the increasing popularity of this entropy in…
Length generalization, defined as the ability to extrapolate from shorter training sequences to longer test ones, is a significant challenge for language models. This issue persists even with large-scale Transformers handling relatively…
We study the expansions of permutation statistics in the basis of functions counting occurrences of a fixed pattern in a permutation. We show the finiteness of these pattern expansions for a class of permutation statistics including the…
Recently, Babson and Steingrimsson (see [BS]) introduced generalized permutations patterns that allow the requirement that two adjacent letters in a pattern must be adjacent in the permutation. We study generating functions for the number…
We consider the problem of producing compact architectures for text classification, such that the full model fits in a limited amount of memory. After considering different solutions inspired by the hashing literature, we propose a method…
Words can be represented by composing the representations of subword units such as word segments, characters, and/or character n-grams. While such representations are effective and may capture the morphological regularities of words, they…
We consider the distribution of cycles in two models of random permutations, that are related to one another. In the first model, cycles receive a weight that depends on their length. The second model deals with permutations of points in…
Every k entries in a permutation can have one of k! different relative orders, called patterns. How many times does each pattern occur in a large random permutation of size n? The distribution of this k!-dimensional vector of pattern…
We apply ideas from the cluster method to q-count the permutations of a multiset according to the number of occurrences of certain generalized patterns, as defined by Babson and Steingrimsson. In particular, we consider those patterns with…
One major deficiency of most semantic representation techniques is that they usually model a word type as a single point in the semantic space, hence conflating all the meanings that the word can have. Addressing this issue by learning…
We investigate a signed version of the Hammersley process, a discrete process on words related to a property of integer sequences called heapability (Byers et al., ANALCO 2011). The specific version that we investigate corresponds to a…
In the context of the genome rearrangement problem, we analyze two well known models, namely the reversal and the prefix reversal models, by exploiting the connection with the notion of permutation pattern. More specifically, for any $k$,…
What can large language models learn? By definition, language models (LM) are distributions over strings. Therefore, an intuitive way of addressing the above question is to formalize it as a matter of learnability of classes of…
Transformer-based large language models (LLMs) are comprised of billions of parameters arranged in deep and wide computational graphs. Several studies on LLM efficiency optimization argue that it is possible to prune a significant portion…
Two same length words are $d$-equivalent if they have same descent set and same underlying alphabet. In particular, two same length permutations are $d$-equivalent if they have same descent set. The popularity of a pattern in a set of words…
In this work, we study properties of texts from the perspective of complex network theory. Words in given texts are linked by co-occurrence and transformed into networks, and we observe that these display topological properties common to…
We review results about the density of typical lattices in $R^n.$ They state that such density is of the order of $2^{-n}.$ We then obtain similar results for random packings in $R^n$: after taking suitably a fraction $\nu$ of a typical…