Related papers: Singular Relative Entropy Coding with Bits-Back Re…
The entropy is one of the most applicable uncertainty measures in many statistical and en- gineering problems. In statistical literature, the entropy is used in calculation of the Kullback- Leibler (KL) information which is a powerful mean…
This paper considers the joint compression of a pair of correlated sources, where the encoder is allowed to access only one of the sources. The objective is to recover both sources under separate distortion constraints for each source while…
Lossy image compression is generally formulated as a joint rate-distortion optimization to learn encoder, quantizer, and decoder. However, the quantizer is non-differentiable, and discrete entropy estimation usually is required for rate…
Random number generators (RNG) based on quantum mechanics are captivating due to their security and unpredictability compared to conventional generators, such as pseudo-random number generators and hardware-random number generators. This…
Entropy measures quantify the amount of information and correlation present in a quantum system. In practice, when the quantum state is unknown and only copies thereof are available, one must resort to the estimation of such entropy…
We study the problem of generating a random variate $X$ from a finite discrete probability distribution $P$ using an entropy source of independent fair coin flips. A classic result from Knuth and Yao shows that the optimal expected number…
One-bit compressed sensing (1bCS) addresses the recovery of sparse signals from highly quantized measurements, retaining only the sign of each linear measurement. In the support recovery setting, the goal is to identify $\text{supp}(x)$,…
In this paper, we prove that the sub-field images of generalized Reed-Solomon (RS) codes can achieve the symmetric capacity of p-ary memoryless channels. Unlike the totally random linear code ensemble, as a class of maximum distance…
We work out a theory of approximate quantum error correction that allows us to derive a general lower bound for the entanglement fidelity of a quantum code. The lower bound is given in terms of Kraus operators of the quantum noise. This…
The problem of 1-bit compressive sampling is addressed in this paper. We introduce an optimization model for reconstruction of sparse signals from 1-bit measurements. The model targets a solution that has the least l0-norm among all signals…
This paper studies the problem of extracting common randomness (CR) or secret keys from correlated random sources observed by two legitimate parties, Alice and Bob, through public discussion in the presence of an eavesdropper, Eve. We…
It is shown that an i.i.d. binary source sequence $X_1, \ldots, X_n$ can be losslessly compressed at any rate above entropy such that the individual decoding of any $X_i$ reveals \emph{no} information about the other bits $\{X_j : j \neq…
Estimating the entropy rate of discrete time series is a challenging problem with important applications in numerous areas including neuroscience, genomics, image processing and natural language processing. A number of approaches have been…
In this paper, we introduce the notion of recurrence entropy in the context of nonlinear control systems. A set is said to be ($\tau$-)recurrent if every trajectory that starts in the set returns to it (within at most $\tau$ units of time).…
We consider a bound on the bias reduction of a random number generator by processing based on binary linear codes. We introduce a new bound on the total variation distance of the processed output based on the weight distribution of the code…
In the emerging field of goal-oriented communications, the focus has shifted from reconstructing data to directly performing specific learning tasks, such as classification, segmentation, or pattern recognition, on the received coded data.…
This paper investigates the general distributed lossless/lossy source coding formulated by Jana and Blahut. Their multi-letter rate-distortion region, an alternative to the region derived by Yang and Qin, is characterized by entropy…
Random linear codes are a workhorse in coding theory, and are used to show the existence of codes with the best known or even near-optimal trade-offs in many noise models. However, they have little structure besides linearity, and are not…
This paper investigates the problem of source-channel coding for secure transmission with arbitrarily correlated side informations at both receivers. This scenario consists of an encoder (referred to as Alice) that wishes to compress a…
We consider error-correction coding schemes for adversarial wiretap channels (AWTCs) in which the channel can a) read a fraction of the codeword bits up to a bound $r$ and b) flip a fraction of the bits up to a bound $p$. The channel can…