Related papers: Lossy Source Compression of Non-Uniform Binary Sou…
Non-adaptive joint source network coding of correlated sources is discussed in this paper. By studying the information flow in the network, we propose quantized network coding as an alternative for packet forwarding. This technique has both…
Efficient and accurate decoding of quantum error-correcting codes is essential for fault-tolerant quantum computation, however, it is challenging due to the degeneracy of errors, the complex code topology, and the large space for logical…
In order to compress quantum messages without loss of information it is necessary to allow the length of the encoded messages to vary. We develop a general framework for variable-length quantum messages in close analogy to the classical…
The problem of joint universal source coding and identification is considered in the setting of fixed-rate lossy coding of continuous-alphabet memoryless sources. For a wide class of bounded distortion measures, it is shown that any…
Data compression is a well-studied (and well-solved) problem in the setup of long coding blocks. But important emerging applications need to compress data to memory words of small fixed widths. This new setup is the subject of this paper.…
How can we quantize large language models while preserving accuracy? Quantization is essential for deploying large language models (LLMs) efficiently. Binary-coding quantization (BCQ) and uniform quantization (UQ) are promising quantization…
We prove the existence of codebooks for d-semifaithful lossy compression that are simultaneously universal with respect to both the class of finite-alphabet memoryless sources and the class of all bounded additive distortion measures. By…
Consider the following distributed optimization scenario. A worker has access to training data that it uses to compute the gradients while a server decides when to stop iterative computation based on its target accuracy or delay…
By looking at quantum data compression in the second quantisation, we present a new model for the efficient generation and use of variable length codes. In this picture lossless data compression can be seen as the {\em minimum energy}…
We propose a random coding technique for joint source-channel coding of discrete memoryless sources and channels. The approach builds on the random Gilbert-Varshamov code construction of Somekh-Baruch et al. and extends it to the joint…
Quantum error correction is rapidly seeing first experimental implementations, but there is a significant gap between asymptotically optimal error-correcting codes and codes that are experimentally feasible. Quantum LDPC codes range from…
Large alphabet source coding is a basic and well-studied problem in data compression. It has many applications such as compression of natural language text, speech and images. The classic perception of most commonly used methods is that a…
We study the compression of LLM-generated text across lossless and lossy regimes, characterizing a compression-compute frontier where more compression is possible at the cost of more compute. For lossless compression, domain-adapted LoRA…
Low-density parity-check (LDPC) codes have been used for communication over a two-user Gaussian broadcast channel. It has been shown in the literature that the optimal decoding of such system requires joint decoding of both user messages at…
In practice, LDPC codes are decoded using message passing methods. These methods offer good performance but tend to converge slowly and sometimes fail to converge and to decode the desired codewords correctly. Recently, tree-reweighted…
We consider the lossy quantum source coding problem where the task is to compress a given quantum source below its von Neumann entropy. Inspired by the duality connections between the rate-distortion and channel coding problems in the…
We propose a novel joint decoding technique for distributed source-channel (DSC) coded systems for transmission of correlated binary Markov sources over additive white Gaussian noise (AWGN) channels. In the proposed scheme, relatively…
We investigate the effects of post-training quantization and quantization-aware training on the generalization of Transformer language models. We present a new method called self-distilled quantization (SDQ) that minimizes accumulative…
We consider the problem of joint universal variable-rate lossy coding and identification for parametric classes of stationary $\beta$-mixing sources with general (Polish) alphabets. Compression performance is measured in terms of…
In this paper, we distinguish two guessing algorithms for decoding binary linear codes. One is the guessing noise decoding (GND) algorithm, and the other is the guessing codeword decoding (GCD) algorithm. We prove that the GCD is a maximum…