Related papers: Asymptotic Optimality of Antidictionary Codes
The following critical phenomenon was recently discovered. When a memoryless source is compressed using a variable-length fixed-distortion code, the fastest convergence rate of the (pointwise) compression ratio to the optimal $R(D)$…
We present new lower and upper bounds for the compression rate of binary prefix codes optimized over memoryless sources according to two related exponential codeword length objectives. The objectives explored here are exponential-average…
This paper presents an axiomatic approach to finite Markov decision processes where the discount rate is zero. One of the principal difficulties in the no discounting case is that, even if attention is restricted to stationary policies, a…
This paper combines data-driven and model-driven methods for real-time misinformation detection. Our algorithm, named QuickStop, is an optimal stopping algorithm based on a probabilistic information spreading model obtained from labeled…
We consider a channel-independent decoder which is for i.i.d. random codes what the maximum mutual-information decoder is for constant composition codes. We show that this decoder results in exactly the same i.i.d. random coding error…
In this paper, we analyze classical data compression with quantum side information (also known as the classical-quantum Slepian-Wolf protocol) in the so-called large and moderate deviation regimes. In the non-asymptotic setting, the…
State-of-the-art language models are autoregressive and operate on subword units known as tokens. Specifically, one must encode the conditioning string into a list of tokens before passing to the language models for next-token prediction.…
We consider the problem of approximating optimal in the Minimum Mean Squared Error (MMSE) sense nonlinear filters in a discrete time setting, exploiting properties of stochastically convergent state process approximations. More…
We consider the problem of constructing prefix-free codes in which a designated symbol, a space, can only appear at the end of codewords. We provide a linear-time algorithm to construct almost-optimal codes with this property, meaning that…
We collect several observations that concern variable-length coding of two-sided infinite sequences in a probabilistic setting. Attention is paid to images and preimages of asymptotically mean stationary measures defined on subsets of these…
We propose a generalization of the asymptotic equipartition property to discrete sources with an ambiguous alphabet, and prove that it holds for irreducible stationary Markov sources with an arbitrary distinguishability relation. Our…
The error exponent of fixed-length lossy source coding was established by Marton. Ahlswede showed that this exponent can be discontinuous at a rate $R$, depending on the probability distribution $P$ of the given information source and the…
The problem of lossless data compression with side information available to both the encoder and the decoder is considered. The finite-blocklength fundamental limits of the best achievable performance are defined, in two different versions…
Compression refers to encoding data using bits, so that the representation uses as few bits as possible. Compression could be lossless: i.e. encoded data can be recovered exactly from its representation) or lossy where the data is…
A likelihood encoder is studied in the context of lossy source compression. The analysis of the likelihood encoder is based on the soft-covering lemma. It is demonstrated that the use of a likelihood encoder together with the soft-covering…
Byte-Pair Encoding (BPE) is a widely used method for subword tokenization, with origins in grammar-based text compression. It is employed in a variety of language processing tasks such as machine translation or large language model (LLM)…
For the classical continuous-time quickest change-point detection problem it is shown that the randomized Shiryaev-Roberts-Pollak procedure is asymptotically nearly minimax-optimal (in the sense of Pollak 1985) in the class of randomized…
The variable-length source coding problem allowing the error probability up to some constant is considered for general sources. In this problem the optimum mean codeword length of variable-length codes has already been determined. On the…
The asymptotic rate vs. distance problem is a long-standing fundamental problem in coding theory. The best upper bound to date was given in 1977 and has received since then numerous proofs and interpretations. Here we provide a new,…
We revisit the source coding problem for a Markov chain under the assumption that the transmission times and how fast the Markov chain transitions its state happen at the same time-scale. Specifically, we assume that the transmission of…