Related papers: Analyzing {\alpha}-divergence in Gaussian Rate-Dis…
A significant bottleneck in federated learning (FL) is the network communication cost of sending model updates from client devices to the central server. We present a comprehensive empirical study of the statistics of model updates in FL,…
In this paper we introduce a definition for nonanticipative Rate Distortion Function (RDF) on abstract alphabets, and we invoke weak convergence of probability measures to show various of its properties, such as, existence of the optimal…
Analog coding decouples the tasks of protecting against erasures and noise. For erasure correction, it creates an "analog redundancy" by means of band-limited discrete Fourier transform (DFT) interpolation, or more generally, by an…
We study the compression of data in the case where the useful information is contained in a set rather than a vector, i.e., the ordering of the data points is irrelevant and the number of data points is unknown. Our analysis is based on…
Consider a Gaussian memoryless multiple source with $m$ components with joint probability distribution known only to lie in a given class of distributions. A subset of $k \leq m$ components are sampled and compressed with the objective of…
In some rate-distortion-type problems, the required fidelity of information is affected by past actions. As a result, the distortion function depends not only on the instantaneous distortion between a source symbol and its representation…
We study the neural network (NN) compression problem, viewing the tension between the compression ratio and NN performance through the lens of rate-distortion theory. We choose a distortion metric that reflects the effect of NN compression…
In this paper, the rate-distortion theory of the Gray-Wyner lossy source coding system is investigated. For the case of jointly Gaussian distributed sources, we establish an expression for the rate-distortion function under the constraint…
The problem of optimal real-time transmission of a Markov source under constraints on the expected number of transmissions is considered, both for the discounted and long term average cases. This setup is motivated by applications where…
This paper studies the recovery of a superposition of point sources from noisy bandlimited data. In the fewest possible words, we only have information about the spectrum of an object in a low-frequency band bounded by a certain cut-off…
Consider a pair of correlated Gaussian sources (X1,X2). Two separate encoders observe the two components and communicate compressed versions of their observations to a common decoder. The decoder is interested in reconstructing a linear…
In this work we investigate the behavior of the minimal rate needed in order to guarantee a given probability that the distortion exceeds a prescribed threshold, at some fixed finite quantization block length. We show that the excess coding…
A computable expression for the rate-distortion (RD) function proposed by Heegard and Berger has eluded information theory for nearly three decades. Heegard and Berger's single-letter achievability bound is well known to be optimal for…
The optimal quantum measurements for estimating individual parameters might be incompatible with each other so that they cannot be jointly performed. The tradeoff between the estimation precision for different parameters can be…
Estimating the angular separation between two incoherently radiating monochromatic point sources is a canonical toy problem to quantify spatial resolution in imaging. In recent work, Tsang {\em et al.} showed, using a Fisher Information…
We present the first unified framework for rate-distortion-optimized compression and segmentation of 3D Gaussian Splatting (3DGS). While 3DGS has proven effective for both real-time rendering and semantic scene understanding, prior works…
We provide exact asymptotic expressions for the performance of regression by an $L-$layer deep random feature (RF) model, where the input is mapped through multiple random embedding and non-linear activation functions. For this purpose, we…
It is well known that independent (separate) encoding of K correlated sources may incur some rate loss compared to joint encoding, even if the decoding is done jointly. This loss is particularly evident in the multiple descriptions problem,…
It is known that circularly symmetric Gaussian signals are the optimal input signals for the partial decode-and-forward (PDF) coding scheme in the Gaussian multiple-input multiple-output (MIMO) relay channel, but there is currently no…
Rate-distortion theory provides bounds for compressing data produced by an information source to a specified encoding rate that is strictly less than the source's entropy. This necessarily entails some loss, or distortion, between the…