Related papers: Bubble-Flip -- A New Generation Algorithm for Pref…
We present a new heuristic algorithm finding reset words. The algorithm called CutOff-IBFS is based on a simple idea of inverse breadth-first-search in the power automaton. We perform an experimental investigation of effectiveness compared…
We present a method which displays all palindromes of a given length from De Bruijn words of a certain order, and also a recursive one which constructs all palindromes of length $n+1$ from the set of palindromes of length $n$. We show that…
Neural text generation is a key tool in natural language applications, but it is well known there are major problems at its core. In particular, standard likelihood training and decoding leads to dull and repetitive outputs. While some…
Grammar-based compression is a popular and powerful approach to compressing repetitive texts but until recently its relatively poor time-space trade-offs during real-life construction made it impractical for truly massive datasets such as…
This paper describes the probabilistic behaviour of a random Sturmian word. It performs the probabilistic analysis of the recurrence function which can be viewed as a waiting time to discover all the factors of length $n$ of the Sturmian…
Indexing strings via prefix (or suffix) sorting is, arguably, one of the most successful algorithmic techniques developed in the last decades. Can indexing be extended to languages? The main contribution of this paper is to initiate the…
Recently, Cenzato et al.\ proposed a new text index, called the \emph{suffixient array}, which is a subset of the suffix array and supports locating a single pattern occurrence or finding its maximal exact matches (MEMs), assuming random…
Feedforward Neural Network (FNN)-based language models estimate the probability of the next word based on the history of the last N words, whereas Recurrent Neural Networks (RNN) perform the same task based only on the last word and some…
We show that the equality language of two non-periodic binary morphisms is generated by at most two words. If its rank is two, then the generators start (and end) with different letters. This in particular implies that any binary language…
A flip-swap language is a set S of binary strings of length n such that $S \cup 0^n$ is closed under two operations (when applicable): (1) Flip the leftmost 1; and (2) Swap the leftmost 1 with the bit to its right. Flip-swap languages model…
Text generation tasks, including translation, summarization, language models, and etc. see rapid growth during recent years. Despite the remarkable achievements, the repetition problem has been observed in nearly all text generation models…
We present three simple algorithms to uniformly generate `Fibonacci words' (i.e., some words that are enumerated by Fibonacci numbers), Schr{\"o}der trees of size $n$ and Motzkin left factors of size $n$ and final height $h$. These…
A deterministic BSP algorithm for constructing the suffix array of a given string is presented, based on a technique which we call accelerated sampling. It runs in optimal O(n/p) local computation and communication, and requires a near…
We describe, analyze, and evaluate experimentally a new probabilistic model for word-sequence prediction in natural language based on prediction suffix trees (PSTs). By using efficient data structures, we extend the notion of PST to…
We present a new graph compressor that works by recursively detecting repeated substructures and representing them through grammar rules. We show that for a large number of graphs the compressor obtains smaller representations than other…
Comprehensively searching for words in Sanskrit E-text is a non-trivial problem because words could change their forms in different contexts. One such context is sandhi or euphonic conjunctions, which cause a word to change owing to the…
Universal Cycles, or U-cycles, as originally defined by de Bruijn, are an efficient method to exhibit a large class of combinatorial objects in a compressed fashion, and with no repeats. de Bruijn's theorem states that U-cycles for $n$…
To guide the generation of large pretrained language models (LM), previous work has focused on directly fine-tuning the language model or utilizing an attribute discriminator. In this work, we propose a novel lightweight framework for…
We show that the number of prefix normal binary words of length $n$ is $2^{n-\Theta((\log n)^2)}$. We also show that the maximum number of binary words of length $n$ with a given fixed prefix normal form is $2^{n-O(\sqrt{n\log n})}$.
First introduced in the study of the Sturmian words, the iterated palindromic closure was recently generalized to pseudopalindromes. This operator allows one to construct words with an infinity of pseudopalindromic prefixes, called…