Related papers: Exponential Strong Converse for Successive Refinem…
A strong converse shows that no procedure can beat the asymptotic (as blocklength $n\to\infty$) fundamental limit of a given information-theoretic problem for any fixed error probability. A second-order converse strengthens this conclusion…
Recently, attention-based transformers have become a de facto standard in many deep learning applications including natural language processing, computer vision, signal processing, etc.. In this paper, we propose a transformer-based…
The problem of joint source-channel coding is considered for a stationary remote (noisy) Gaussian source and a Gaussian channel. The encoder and decoder are assumed to be causal and their combined operations are subject to a delay…
In this paper, distributed (or multiterminal) source coding with one distortion criterion and correlated messages is considered. This problem can be also called ``Berger-Yeung problem with correlated messages''. It corresponds to the source…
As conventional communication systems based on classic information theory have closely approached the limits of Shannon channel capacity, semantic communication has been recognized as a key enabling technology for the further improvement of…
In this first part, a computable outer bound is proved for the multiterminal source coding problem, for a setup with two encoders, discrete memoryless sources, and bounded distortion measures.
We construct a new family of explicit codes that are list decodable to capacity and achieve an optimal list size of $O(\frac{1}{\epsilon})$. In contrast to existing explicit constructions of codes achieving list decoding capacity, our…
We prove the following results concerning the list decoding of error-correcting codes: (i) We show that for \textit{any} code with a relative distance of $\delta$ (over a large enough alphabet), the following result holds for \textit{random…
In this work we investigate the behavior of the minimal rate needed in order to guarantee a given probability that the distortion exceeds a prescribed threshold, at some fixed finite quantization block length. We show that the excess coding…
I introduce rate-distortion theory for quantum coding, and derive a lower bound, involving the coherent information, on the rate at which qubits must be used to encode a quantum source with a given maximum level of distortion per source…
Consider a discrete memoryless multiple source with $m$ components of which $k \leq m$ possibly different sources are sampled at each time instant and jointly compressed in order to reconstruct all the $m$ sources under a given distortion…
In this paper, we consider a distributed remote source coding problem, where a sequence of observations of source vectors is available at the encoder. The problem is to specify the optimal rate for encoding the observations subject to a…
Inspired by an interesting idea of Cai and Zhang, we formulate and prove the convex $k$-sparse decomposition of vectors which is invariant with respect to $\ell_1$ norm. This result fits well in discussing compressed sensing problems under…
Recently, Tian et al. [1] considered joint source-channel coding of transmitting a Gaussian source over $K$-user Gaussian broadcast channel, and derived an outer bound on the admissible distortion region. In [1], they stated "due to its…
We consider an inverse problem of reconstructing a degeneracy point in the diffusion coefficient in a one-dimensional parabolic equation by measuring the normal derivative on one side of the domain boundary. We analyze the sensitivity of…
The sequence reconstruction problem, introduced by Levenshtein in 2001, considers a communication setting in which a sender transmits a codeword and the receiver observes K independent noisy versions of this codeword. In this work, we study…
We study the problem of exact sampling under an exponential communication cost, specifically Campbell's average codeword length $L(t)$ of order $t$, and R\'enyi's entropy. We provide a lower bound on the Campbell cost of exact sampling that…
In the Wyner-Ziv source coding problem, a source $X$ has to be encoded while the decoder has access to side information $Y$. This paper investigates the indirect setup, in which a latent source $S$, unobserved by both the encoder and the…
In image compression, with recent advances in generative modeling, the existence of a trade-off between the rate and the perceptual quality has been brought to light, where the perception is measured by the closeness of the output…
Top-$k$ decoding is a widely used method for sampling from LLMs: at each token, only the largest $k$ next-token-probabilities are kept, and the next token is sampled after re-normalizing them to sum to unity. Top-$k$ and other sampling…