Related papers: An algorithm for the word entropy
A word $u$ is a subsequence of another word $w$ if $u$ can be obtained from $w$ by deleting some of its letters. The word $w$ with alph$(w)=\Sigma$ is called $k$-subsequence universal if the set of subsequences of length $k$ of $w$ contains…
We analyze the algorithm in [Holub, 2009], which decides whether a given word is a fixed point of a nontrivial morphism. We show that it can be implemented to have complexity in O(mn), where n is the length of the word and m the size of the…
Word embedding, specially with its recent developments, promises a quantification of the similarity between terms. However, it is not clear to which extent this similarity value can be genuinely meaningful and useful for subsequent tasks.…
The aim of this note is to provide some reference facts for LZW---mostly from Thomas and Cover \cite{Cover:2006aa} and provide a reference for some metrics that can be derived from it. LZW is an algorithm to compute a Kolmogorov Complexity…
The random Fibonacci chain is a generalisation of the classical Fibonacci substitution and is defined as the rule mapping $0\mapsto 1$ and $1 \mapsto 01$ with probability $p$ and $1 \mapsto 10$ with probability $1-p$ for $0<p<1$ and where…
We introduce the notions of topological entropy of a formal language and of a topological automaton. We show that the entropy function is surjective and bound the entropy of languages accepted by deterministic {\epsilon}-free push-down…
A closed word (a.k.a. periodic-like word or complete first return) is a word whose longest border does not have internal occurrences, or, equivalently, whose longest repeated prefix is not right special. We investigate the structure of…
Given a knowledge base KB containing first-order and statistical facts, we consider a principled method, called the random-worlds method, for computing a degree of belief that some formula Phi holds given KB. If we are reasoning about a…
A non-empty word $w$ is a \emph{border} of a word $u$ if $\vert w\vert<\vert u\vert$ and $w$ is both a prefix and a suffix of $u$. A word $u$ is \emph{privileged} if $\vert u\vert\leq 1$ or if $u$ has a privileged border $w$ that appears…
Understanding the complexity of human language requires an appropriate analysis of the statistical distribution of words in texts. We consider the information retrieval problem of detecting and ranking the relevant words of a text by means…
A subsequence of a word $w$ is a word $u$ such that $u = w[i_1] w[i_2] \dots w[i_{k}]$, for some set of indices $1 \leq i_1 < i_2 < \dots < i_k \leq \lvert w\rvert$. A word $w$ is $k$-subsequence universal over an alphabet $\Sigma$ if every…
These lecture notes provide an introduction to combinatorics on words and its interactions with dynamics, algebra, and arithmetic. The central theme is the notion of low factor complexity for infinite words. We investigate the following…
We estimate the $n$-gram entropies of natural language texts in word-length representation and find that these are sensitive to text language and genre. We attribute this sensitivity to changes in the probability distribution of the lengths…
Entropies must correspond to mean values for them to be measurable. The Shannon entropy corresponds to the weighted arithmetic mean, whereas the Renyi entropy corresponds to the exponential mean. These means refer to code lengths, which are…
Given a countable relational language $L$, we consider probability measures on the space of $L$-structures with underlying set $\mathbb{N}$ that are invariant under the logic action. We study the growth rate of the entropy function of such…
In 2005, Rampersad and the second author proved a number of theorems about infinite words x with the property that if w is any sufficiently long finite factor of x, then its reversal w^R is not a factor of x. In this note we revisit these…
For a finite word $w$ we define and study the Kolmogorov structure function $h_w$ for nondeterministic automatic complexity. We prove upper bounds on $h_w$ that appear to be quite sharp, based on numerical evidence.
Borel and Reutenauer (2006) showed, \emph{inter alia}, that a word $w$ of length $n>1$ is conjugate to a Christoffel word if and only if for $k=0,1, \dots , n-1$, $w$ has $k+1$ distinct circular factors of length $k$. Sturmian words are the…
We extend the notion of estimation entropy of autonomous dynamical systems proposed by Liberzon and Mitra [1] to nonlinear dynamical systems with uncertain inputs with bounded variation. We call this new notion the {$\epsilon$}-estimation…
In this paper we study certain properties of R\'{e}nyi entropy functionals $H_\alpha(\mathcal{P})$ on the space of probability distributions over $\mathbb{Z}_+$. Primarily, continuity and convergence issues are addressed. Some properties…