Converse Bounds for Entropy-Constrained Quantization Via a Variational Entropy Inequality
Abstract
We derive a lower bound on the smallest output entropy that can be achieved via vector quantization of a -dimensional source with given expected th-power distortion. Specialized to the one-dimensional case, and in the limit of vanishing distortion, this lower bound converges to the output entropy achieved by a uniform quantizer, thereby recovering the result by Gish and Pierce that uniform quantizers are asymptotically optimal as the allowed distortion tends to zero. Our lower bound holds for all -dimensional memoryless sources having finite differential entropy and whose integer part has finite entropy. In contrast to Gish and Pierce, we do not require any additional constraints on the continuity or decay of the source probability density function. For one-dimensional sources, the derivation of the lower bound reveals a necessary condition for a sequence of quantizers to be asymptotically optimal as the allowed distortion tends to zero. This condition implies that any sequence of asymptotically-optimal almost-regular quantizers must converge to a uniform quantizer as the allowed distortion tends to zero.
Cite
@article{arxiv.1507.08349,
title = {Converse Bounds for Entropy-Constrained Quantization Via a Variational Entropy Inequality},
author = {Tobias Koch and Gonzalo Vazquez-Vilar},
journal= {arXiv preprint arXiv:1507.08349},
year = {2017}
}
Comments
26 pages, 1 figure. Submitted to IEEE Transactions on Information Theory. Most important changes with respect to previous version: i) changed title (the old title was "Rate-distortion bounds for high-resolution vector quantization via Gibbs's inequality"); ii) added necessary conditions for a sequence of quantizers to be asymptotically optimal (Theorem 7 and Corollary 8)