Related papers: Towards Optimality in Transform Coding
An effective 'on-the-fly' mechanism for stochastic lossy coding of Markov sources using string matching techniques is proposed in this paper. Earlier work has shown that the rate-distortion bound can be asymptotically achieved by a 'natural…
We consider a distributed source coding problem of $L$ correlated Gaussian observations $Y_i, i=1,2,...,L$. We assume that the random vector $Y^{L}={}^{\rm t} (Y_1,Y_2,$ $...,Y_L)$ is an observation of the Gaussian random vector…
In this work, the problem of transmitting an i.i.d Gaussian source over an i.i.d Gaussian wiretap channel with an i.i.d Gaussian side information is considered. The intended receiver is assumed to have a certain minimum SNR and the…
We construct an optimal quantum universal variable-length code that achieves the admissible minimum rate, i.e., our code is used for any probability distribution of quantum states. Its probability of exceeding the admissible minimum rate…
The optimal causal coding of a partially observed Markov process is studied, where the cost to be minimized is a bounded, non-negative, additive, measurable single-letter function of the source and the receiver output. A structural result…
Conditional Optimal Transport (COT) problem aims to find a transport map between conditional source and target distributions while minimizing the transport cost. Recently, these transport maps have been utilized in conditional generative…
In this paper, we obtain necessary optimality conditions for neural network approximation. We consider neural networks in Manhattan ($l_1$ norm) and Chebyshev ($\max$ norm). The optimality conditions are based on neural networks with at…
In this paper we consider the minimization of a continuous function that is potentially not differentiable or not twice differentiable on the boundary of the feasible region. By exploiting an interior point technique, we present first- and…
For the last few decades, the application of signal-adaptive transform coding to video compression has been stymied by the large computational complexity of matrix-based solutions. In this paper, we propose a novel parametric approach to…
This is a tutorial and survey paper on Karush-Kuhn-Tucker (KKT) conditions, first-order and second-order numerical optimization, and distributed optimization. After a brief review of history of optimization, we start with some preliminaries…
The classical method to solve a quadratic optimization problem with nonlinear equality constraints is to solve the Karush-Kuhn-Tucker (KKT) optimality conditions using Newton's method. This approach however is usually computationally…
In this paper, we study systematic Luby Transform (SLT) codes over additive white Gaussian noise (AWGN) channel. We introduce the encoding scheme of SLT codes and give the bipartite graph for iterative belief propagation (BP) decoding…
We investigate the upper and lower bounds on the quantization distortions for independent and identically distributed sources in the finite block-length regime. Based on the convex optimization framework of the rate-distortion theory, we…
The distortion-rate function of output-constrained lossy source coding with limited common randomness is analyzed for the special case of squared error distortion measure. An explicit expression is obtained when both source and…
The classical coding theorem in Kolmogorov complexity states that if an $n$-bit string $x$ is sampled with probability $\delta$ by an algorithm with prefix-free domain then K$(x) \leq \log(1/\delta) + O(1)$. In a recent work, Lu and…
We introduce a general framework for end-to-end optimization of the rate--distortion performance of nonlinear transform codes assuming scalar quantization. The framework can be used to optimize any differentiable pair of analysis and…
Analog (uncoded) transmission provides a simple and robust scheme for communicating a Gaussian source over a Gaussian channel under the mean squared error (MSE) distortion measure. Unfortunately, its performance is usually inferior to the…
This paper investigates the statistical properties of non-linear transformations (NLT) of random variables, in order to establish useful tools for estimation and information theory. Specifically, the paper focuses on linear regression…
The rate-distortion curve captures the fundamental tradeoff between compression length and resolution in lossy data compression. However, it conceals the underlying dynamics of optimal source encodings or test channels. We argue that these…
Optimal transport has numerous applications, particularly in machine learning tasks involving generative models. In practice, the transportation process often encounters an information bottleneck, typically arising from the conversion of a…