Related papers: The length of a typical Huffman codeword
We model the generation of words with independent unequal probabilities of occurrence of letters. We prove that the probability $p(r)$ of occurrence of words of rank $r$ has a power asymptotics. As distinct from the paper published earlier…
We show that for a wide range of channels and code ensembles with pairwise-independent codewords, with probability tending to 1 with the code length, expurgating an arbitrarily small fraction of codewords from a randomly selected code…
Probability estimation is an elementary building block of every statistical data compression algorithm. In practice probability estimation is often based on relative letter frequencies which get scaled down, when their sum is too large.…
The combined universal probability M(D) of strings x in sets D is close to max_{x \in D} M({x}): their ~ logs differ by at most D's information j = I(D:H) about the halting sequence H. Thus if all x have complexity K(x) > k, D carries > i…
Consider the set of source distributions within a fixed maximum relative entropy with respect to a given nominal distribution. Lossless source coding over this relative entropy ball can be approached in more than one way. A problem…
The Huffman coding algorithm is interpreted in the lattice of partitions of the source alphabet. Maximal chains in the partition lattice correspond to linear extensions of tree orders, and those among the chains that exhibit a simple greedy…
There is a common problem of operating on hash values of elements of some database. In this paper there will be analyzed informational content of such general task and how to practically approach such found lower boundaries. Minimal prefix…
We derive a single-letter upper bound to the mismatched-decoding capacity for discrete memoryless channels. The bound is expressed as the mutual information of a transformation of the channel, such that a maximum-likelihood decoding error…
The relationship between the quality of a string, as judged by a human reader, and its probability, $p(\boldsymbol{y})$ under a language model undergirds the development of better language models. For example, many popular algorithms for…
The normalized maximum likelihood code length has been widely used in model selection, and its favorable properties, such as its consistency and the upper bound of its statistical risk, have been demonstrated. This paper proposes a novel…
Let x, y be strings of equal length. The Hamming distance h(x,y) between x and y is the number of positions in which x and y differ. If x is a cyclic shift of y, we say x and y are conjugates. We consider f(x,y), the Hamming distance…
We study universal compression of sequences generated by monotonic distributions. We show that for a monotonic distribution over an alphabet of size $k$, each probability parameter costs essentially $0.5 \log (n/k^3)$ bits, where $n$ is the…
AIFV (almost instantaneous fixed-to-variable length) codes are noiseless source codes that can attain a shorter average codeword length than Huffman codes by allowing a time-variant encoder with two code tables and a decoding delay of at…
Let $\mathcal{P}_n$ be the set of all probability mass functions (PMFs) $(p_1,p_2,\ldots,p_n)$ that satisfy $p_i>0$ for $1\leq i \leq n$. Define the minimum expected length function $\mathcal{L}_D :\mathcal{P}_n \rightarrow \mathbb{R}$ such…
The prefix palindromic length $PPL_u(n)$ of an infinite word $u$ is the minimal number of concatenated palindromes needed to express the prefix of length $n$ of $u$. In a 2013 paper with Puzynina and Zamboni we stated the conjecture that…
The distributions of the $m$-th longest runs of multivariate random sequences are considered. For random sequences made up of $k$ kinds of letters, the lengths of the runs are sorted in two ways to give two definitions of run length…
Consider the following Markov chain on permutations of length $n$. At each time step we choose a random position. If the letter at that position is smaller than the letter immediately to the left (cyclically) then these letters swap…
Suppose that, for any (k \geq 1), (\epsilon > 0) and sufficiently large $\sigma$, we are given a black box that allows us to sample characters from a $k$th-order Markov source over the alphabet (\{0, ..., \sigma - 1\}). Even if we know the…
This paper considers the problem of variable-length lossy source coding. The performance criteria are the excess distortion probability and the cumulant generating function of codeword lengths. We derive a non-asymptotic fundamental limit…
This paper investigates the problem of variable-length lossy source coding allowing a positive excess distortion probability and an overflow probability of codeword lengths. Novel one-shot achievability and converse bounds of the optimal…