Related papers: On scattered subword complexity
The problem addressed concerns the determination of the average number of successive attempts of guessing a word of a certain length consisting of letters with given probabilities of occurrence. Both first- and second-order approximations…
Words in natural language follow a Zipfian distribution whereby some words are frequent but most are rare. Learning representations for words in the "long tail" of this distribution requires enormous amounts of data. Representations of rare…
How many words are needed to define all the words in a dictionary? Graph-theoretic analysis reveals that about 10% of a dictionary is a unique Kernel of words that define one another and all the rest, but this is not the smallest such…
We fully classify automatic sequences $a$ over a finite alphabet $\Omega$ with the property that each word over $\Omega$ appears is $a$ along an arithmetic progression. Using the terminology introduced by Avgustinovich, Fon-Der-Flaass and…
The downward closure of a word language is the set of all (not necessarily contiguous) subwords of its members. It is well-known that the downward closure of any language is regular. While the downward closure appears to be a powerful…
We relate the notions of spectral gap for unitary representations and subfactors with definability of certain important sets in the corresponding structures. We give several applications of this relationship.
Language models (LMs) estimate a probability distribution over strings in a natural language; these distributions are crucial for computing perplexity and surprisal in linguistics research. While we are usually concerned with measuring…
In this article, we study subword complexity of colorings of regular trees. We characterize colorings of bounded subword complexity and study Sturmian colorings, which are colorings of minimal unbounded subword complexity. We classify…
Formula for the size of the scatterer is derived explicitly in terms of the scattering amplitude corresponding to this scatterer. By the scatterer either a bounded obstacle $D$ or the support of the compactly supported potential is meant
Modern natural language understanding models depend on pretrained subword embeddings, but applications may need to reason about words that were never or rarely seen during pretraining. We show that examples that depend critically on a rarer…
Given a countable set X (usually taken to be N or Z), an infinite permutation $\pi$ of X is a linear ordering $<_\pi$ of X. This paper investigates the combinatorial complexity of infinite permutations on N associated with the image of…
An evaluation of distributed word representation is generally conducted using a word similarity task and/or a word analogy task. There are many datasets readily available for these tasks in English. However, evaluating distributed…
The automatic disambiguation of word senses (i.e., the identification of which of the meanings is used in a given context for a word that has multiple meanings) is essential for such applications as machine translation and information…
We prove that the isomorphism of scattered tree automatic linear orders as well as the existence of automorphisms of scattered word automatic linear orders are undecidable. For the existence of automatic automorphisms of word automatic…
The height of a piecewise-testable language $L$ is the maximum length of the words needed to define $L$ by excluding and requiring given subwords. The height of $L$ is an important descriptive complexity measure that has not yet been…
Given a countable set X (usually taken to be the natural numbers or integers), an infinite permutation, \pi, of X is a linear ordering of X. This paper investigates the combinatorial complexity of infinite permutations on the natural…
We present a model of speech perception which takes into account effects of correlations between sounds. Words in this model correspond to the attractors of a suitably chosen descent dynamics. The resulting lexicon is rich in short words,…
In this paper we explore a new hierarchy of classes of languages and infinite words and its connection with complexity classes. Namely, we say that a language belongs to the class $L_k$ if it is a subset of the catenation of $k$ languages…
We investigate the number of sets of words that can be formed from a finite alphabet, counted by the total length of the words in the set. An explicit expression for the counting sequence is derived from the generating function, and…
I explore some of the issues that arise when trying to establish a connection between the underspecification hypothesis pursued in the NLP literature and work on ambiguity in semantics and in the psychological literature. A theory of…