Related papers: Message passing algorithms for non-linear nodes an…
We study the neural network (NN) compression problem, viewing the tension between the compression ratio and NN performance through the lens of rate-distortion theory. We choose a distortion metric that reflects the effect of NN compression…
Autoencoders are a prominent model in many empirical branches of machine learning and lossy data compression. However, basic theoretical questions remain unanswered even in a shallow two-layer setting. In particular, to what degree does a…
Data-driven artificial intelligence (AI) techniques are becoming prominent for learning in support of data compression, but are focused on standard problems such as text compression. To instead address the emerging problem of semantic…
The subject of this paper is transmission over a general class of binary-input memoryless symmetric channels using error correcting codes based on sparse graphs, namely low-density generator-matrix and low-density parity-check codes. The…
The paper presents bounds on the achievable rates and the decoding complexity of low-density parity-check (LDPC) codes. It is assumed that the communication of these codes takes place over statistically independent parallel channels where…
We define two algorithms for propagating information in classification problems with pairwise relationships. The algorithms are based on contraction maps and are related to non-linear diffusion and random walks on graphs. The approach is…
We investigate an encoding scheme for lossy compression of a binary symmetric source based on simple spatially coupled Low-Density Generator-Matrix codes. The degree of the check nodes is regular and the one of code-bits is Poisson…
The generalization of Shannon's theory to include messages with given autocorrelations is presented. The analytical calculation of the channel capacity is based on the transfer matrix method of the effective 1D Hamiltonian. This bridge…
In distributed optimization and machine learning, multiple nodes coordinate to solve large problems. To do this, the nodes need to compress important algorithm information to bits so that it can be communicated over a digital channel. The…
The problem of error correction for Gallager's low-density parity-check codes is famously equivalent to that of computing marginal Boltzmann probabilities for an Ising-like model with multispin interactions in a non-uniform magnetic field.…
We present a two-step decoder for the parity code and evaluate its performance in code-capacity and faulty-measurement settings. For noiseless measurements, we find that the decoding problem can be reduced to a series of repetition codes…
I investigate the generic problem of lossy compression of a fluctuating stochastic signal $X$ into a discrete representation $Z$ through optimal thresholding. The signal modulates transition rates of a two-state system described by a binary…
Path planning is an important component of au- tonomous mobile sensing systems. This paper studies upper and lower bounds of communication performance over Gaussian sen- sor networks, to drive power-distortion metrics for path planning…
The principal obstacle to quantum information processing with many qubits is decoherence. One source of decoherence is spontaneous emission which causes loss of energy and information. Inability to control system parameters with high…
The inherent degeneracy of quantum low-density parity-check codes poses a challenge to their decoding, as it significantly degrades the error-correction performance of classical message-passing decoders. To improve their performance, a…
This thesis concerns sequential-access data compression, i.e., by algorithms that read the input one or more times from beginning to end. In one chapter we consider adaptive prefix coding, for which we must read the input character by…
Today's networks are controlled assuming pre-compressed and packetized data. For video, this assumption of data packets abstracts out one of the key aspects - the lossy compression problem. Therefore, first, this paper develops a framework…
This paper takes a rate-distortion approach to understanding the information-theoretic laws governing cache-aided communications systems. Specifically, we characterise the optimal tradeoffs between the delivery rate, cache capacity and…
We investigate the stopping redundancy hierarchy of linear block codes and its connection to permutation decoding techniques. An element in the ordered list of stopping redundancy values represents the smallest number of possibly linearly…
We consider the problem of distributed lossy linear function computation in a tree network. We examine two cases: (i) data aggregation (only one sink node computes) and (ii) consensus (all nodes compute the same function). By quantifying…