Related papers: The length of a typical Huffman codeword
It is known that there are infinite words over finite alphabets with Abelian repetition threshold arbitrarily close to 1; however, the construction previously used involves huge alphabets. In this note we give a short cyclic morphism…
We consider a general class of super-additive scores measuring the similarity of two independent sequences of $n$ i.i.d. letters from a finite alphabet. Our object of interest is the mean score by letter $l_n$. By the subadditivity $l_n$ is…
The penalty incurred by imposing a finite delay constraint in lossless source coding of a memoryless source is investigated. It is well known that for the so-called block-to-variable and variable-to-variable codes, the redundancy decays at…
It is argued that there are characteristic intervals associated with any particle that can be derived without reference to the speed of light $c$. Such intervals are inferred from zeros of wavefunctions which are solutions to the…
We evaluate the influence of different alphabet orderings on the Lyndon factorization of a string. Experiments with Pizza & Chili datasets show that for most alphabet reorderings, the number of Lyndon factors is usually small, and the…
We show that the number of $t$-ary trees with path length equal to $p$ is $\exp(h(t^{-1})t\log t \frac{p}{\log p}(1+o(1)))$, where $\entropy(x){=}{-}x\log x {-}(1{-}x)\log (1{-}x)$ is the binary entropy function. Besides its intrinsic…
Language models (LMs) estimate a probability distribution over strings in a natural language; these distributions are crucial for computing perplexity and surprisal in linguistics research. While we are usually concerned with measuring…
The halting probabilities of universal prefix-free machines are universal for the class of reals with computably enumerable left cut (also known as left-c.e. reals), and coincide with the Martin-Loef random elements of this class. We study…
We simply construct a quantum universal variable-length source code in which, independent of information source, both of the average error and the probability that the coding rate is greater than the entropy rate $H(rho_p)$, tend to 0. If…
Given two random finite sequences from $[k]^n$ such that a prefix of the first sequence is a suffix of the second, we examine the length of their longest common subsequence. If $\ell$ is the length of the overlap, we prove that the expected…
A $c$-short program for a string $x$ is a description of $x$ of length at most $C(x) + c$, where $C(x)$ is the Kolmogorov complexity of $x$. We show that there exists a randomized algorithm that constructs a list of $n$ elements that…
When an i.i.d.\ sequence of letters is cut into words according to i.i.d.\ renewal times, an i.i.d.\ sequence of words is obtained. In the \emph{annealed} LDP (large deviation principle) for the empirical process of words, the rate function…
How hard is it guess a password? Massey showed that that the Shannon entropy of the distribution from which the password is selected is a lower bound on the expected number of guesses, but one which is not tight in general. In a series of…
Huffman Compression, also known as Huffman Coding, is one of many compression techniques in use today. The two important features of Huffman coding are instantaneousness that is the codes can be interpreted as soon as they are received and…
Let $\varepsilon>0$ and, for an odd prime $p$, set $$ S_\ell(p):=\sum_{n\le \ell}\left(\frac{n}{p}\right). $$ Define the first-passage time $$ f_\varepsilon(p):=\min\{\ell\ge 1:\ S_\ell(p)<\varepsilon\ell\}. $$ We prove that there exists a…
Given a collection of strings, each with an associated probability of occurrence, the guesswork of each of them is their position in a list ordered from most likely to least likely, breaking ties arbitrarily. Guesswork is central to several…
Given a probability distribution over a set of n words to be transmitted, the Huffman Coding problem is to find a minimal-cost prefix free code for transmitting those words. The basic Huffman coding problem can be solved in O(n log n) time…
We study the membership problem to context-free languages L (CFLs) on probabilistic words, that specify for each position a probability distribution on the letters (assuming independence across positions). Our task is to compute, given a…
We define the ``shift-match number'' for a binary string and we compute the probability of occurrence of a given string as a subsequence in longer strings in terms of its shift-match number. We thus prove that the string matching…
The combined universal probability $\mathbf{m}(D)$ of strings $x$ in sets $D$ is close to max $\mathbf{m}(x)$ over $x$ in $D$: their logs differ by at most $D$'s information $\mathbf{I}(D:\mathcal{H})$ about the halting sequence…