Related papers: An algorithm for the word entropy
We compute the exact maximum state complexity for the language consisting of $m$ words of length $N$, and characterize languages achieving the maximum. We also consider a special case, namely languages $C(w)$ consisting of the conjugates of…
The downward closure of a word language is the set of all (not necessarily contiguous) subwords of its members. It is well-known that the downward closure of any language is regular. While the downward closure appears to be a powerful…
We study equations in groups G with unique m-th roots for each positive integer m. A word equation in two letters is an expression of the form w(X,A) = B, where w is a finite word in the alphabet {X,A}. We think of A,B in G as fixed…
In this paper, we investigate the problem of synthesizing computable functions of infinite words over an infinite alphabet (data omega-words). The notion of computability is defined through Turing machines with infinite inputs which can…
Kitaev, Liese, Remmel, and Sagan recently defined generalized factor order on words comprised of letters from a partially ordered set $(P, \leq_P)$ by setting $u \leq_P w$ if there is a subword $v$ of $w$ of the same length as $u$ such that…
We introduce the Insertion Chain Complex, a higher-dimensional extension of insertion graphs, as a new framework for analyzing finite sets of words. We study its topological and combinatorial properties, in particular its homology groups,…
This paper is the extended version of On the Complexity of Infinite Advice Strings (ICALP 2018). We investigate a notion of comparison between infinite strings. In a general way, if M is a computation model (e.g. Turing machines) and C a…
If w is a word in d>1 letters and G is a finite group, evaluation of w on a uniformly randomly chosen d-tuple in G gives a random variable with values in G, which may or may not be uniform. It is known that if G ranges over finite simple…
We design, implement and test a simple algorithm which computes the approximate entropy of a finite binary string of arbitrary length. The algorithm uses a weighted average of the Shannon Entropies of the string and all but the last binary…
Written language is complex. A written text can be considered an attempt to convey a meaningful message which ends up being constrained by language rules, context dependence and highly redundant in its use of resources. Despite all these…
Generalizing the notion of automatic complexity of individual strings due to Shallit and Wang, we define the automatic complexity $A(E)$ of an equivalence relation $E$ on a finite set $S$ of strings. We prove that the problem of determining…
We compared entropy for texts written in natural languages (English, Spanish) and artificial languages (computer software) based on a simple expression for the entropy as a function of message length and specific word diversity. Code text…
Frid, Puzynina and Zamboni (2013) defined the palindromic length of a finite word $w$ as the minimal number of palindromes whose concatenation is equal to $w$. For an infinite word $u$ we study $PL_{u}$, that is, the function that assigns…
Language prediction is constrained by informational entropy intrinsic to language, such that there exists a limit to how accurate any language model can become and equivalently a lower bound to language compression. The most efficient…
Estimating the p-th frequency moment of data stream is a very heavily studied problem. The problem is actually trivial when p = 1, assuming the strict Turnstile model. The sample complexity of our proposed algorithm is essentially O(1) near…
The Frank-Wolfe algorithm has seen a resurgence in popularity due to its ability to efficiently solve constrained optimization problems in machine learning and high-dimensional statistics. As such, there is much interest in establishing…
In natural language processing, the entropy of a language is a measure of its unpredictability and complexity. The first study on this subject was conducted by Claude Shannon in 1951. By having participants predict the next character in a…
The entropy rate of printed English is famously estimated to be about one bit per character, a benchmark that modern large language models (LLMs) have only recently approached. This entropy rate implies that English contains nearly 80…
Among other things, we prove that, for a doubling weight $w$, $0< p\leq\infty$, $r\in{\mathbb N}_0$, and $0<\alpha <r+1 - 1/\lambda_p$, we have \[ E_n(f)_{p, w_n} = O(n^{-\alpha}) \iff \omega_\varphi^{r+1}(f, n^{-1})_{p, w_n} =…
We consider an input $x$ generated by an unknown stationary ergodic source $X$ that enters a signal processing system $J$, resulting in $w=J(x)$. We observe $w$ through a noisy channel, $y=z(w)$; our goal is to estimate x from $y$, $J$, and…