Related papers: Estimating the Rate-Distortion Function by Wassers…
We extend the Rate-Distortion-Perception (RDP) framework to the R\'enyi information-theoretic regime, utilizing Sibson's $\alpha$-mutual information to characterize the fundamental limits under distortion and perception constraints. For…
Particle-based variational inference offers a flexible way of approximating complex posterior distributions with a set of particles. In this paper we introduce a new particle-based variational inference method based on the theory of…
Deep learning-based image compression algorithms typically focus on designing encoding and decoding networks and improving the accuracy of entropy model estimation to enhance the rate-distortion (RD) performance. However, few algorithms…
For a Gaussian source under mean-squared error (MSE), classical transform coding is rate--distortion (RD) optimal: the Karhunen--Loeve transform (KLT) diagonalizes the covariance, reverse waterfilling allocates the bits, and scalar…
In this paper, we use tools from rate-distortion theory to establish new upper bounds on the generalization error of statistical distributed learning algorithms. Specifically, there are $K$ clients whose individually chosen models are…
Distributionally Robust Optimization (DRO) provides a framework for decision-making under distributional uncertainty, yet its effectiveness can be compromised by outliers in the training data. This paper introduces a principled approach to…
Living organisms rely on internal models of the world to act adaptively. These models, because of resource limitations, cannot encode every detail and hence need to compress information. From a cognitive standpoint, information compression…
Today's networks are controlled assuming pre-compressed and packetized data. For video, this assumption of data packets abstracts out one of the key aspects - the lossy compression problem. Therefore, first, this paper develops a framework…
Many tasks in machine learning and signal processing can be solved by minimizing a convex function of a measure. This includes sparse spikes deconvolution or training a neural network with a single hidden layer. For these problems, we study…
We derive an algorithm for compression of the currents and varifolds representations of shapes, using ridge leverage score (RLS) sampling, and the theory of Nystrom approximation in Reproducing Kernel Hilbert Spaces. Our method is faster…
We describe an image compression method, consisting of a nonlinear analysis transformation, a uniform quantizer, and a nonlinear synthesis transformation. The transforms are constructed in three successive stages of convolutional linear…
Variational principles for the rate distortion (RD) theory in lossy compression are formulated within the ambit of the generalized nonextensive statistics of Tsallis, for values of the nonextensivity parameter satisfying $ 0 < q < 1 $ and $…
We consider a multiterminal source coding problem in which a source is estimated at a central processing unit from lossy-compressed remote observations. Each lossy-encoded observation is produced by a remote sensor which obtains a noisy…
This paper provides a unified treatment to the recovery of structured signals living in a star-shaped set from general quantized measurements $\mathcal{Q}(\mathbf{A}\mathbf{x}-\mathbf{\tau})$, where $\mathbf{A}$ is a sensing matrix,…
Large deep learning models such as BERT and ResNet achieve state-of-the-art performance but are costly to deploy at the edge due to their size and compute demands. We present RMT-KD, a compression method that leverages Random Matrix Theory…
Despite a short history, neural image codecs have been shown to surpass classical image codecs in terms of rate-distortion performance. However, most of them suffer from significantly longer decoding times, which hinders the practical…
For a required payload, the existing reversible data hiding (RDH) methods always expect to reduce the embedding distortion as much as possible, such as by utilizing a well-designed predictor, taking into account the carrier-content…
Marton's optimal error exponent for the lossy source coding problem is defined as a non-convex optimization problem. This fact had prevented us to develop an efficient algorithm to compute it. This problem is caused by the fact that the…
This paper investigates the problem of computing capacity-cost (C-C) functions for continuous channels. Motivated by the Kullback-Leibler divergence (KLD) proximal reformulation of the classical Blahut-Arimoto (BA) algorithm, the…
We consider large scale distributed optimization over a set of edge devices connected to a central server, where the limited communication bandwidth between the server and edge devices imposes a significant bottleneck for the optimization…