The Lack of Convexity of the Relevance-Compression Function
Information Theory
2022-04-26 v1 math.IT
Abstract
In this paper we investigate the convexity of the relevance-compression function for the Information Bottleneck and the Information Distortion problems. This curve is an analog of the rate-distortion curve, which is convex. In the problems we discuss in this paper, the distortion function is not a linear function of the quantizer, and the relevance-compression function is not necessarily convex (concave), but can change its convexity. We relate this phenomena with existence of first order phase transitions in the corresponding Lagrangian as a function of the annealing parameter.
Keywords
Cite
@article{arxiv.2204.10957,
title = {The Lack of Convexity of the Relevance-Compression Function},
author = {Albert E Parker and Tomas Gedeon and Alexander Dimitrov},
journal= {arXiv preprint arXiv:2204.10957},
year = {2022}
}
Comments
8 pages, 2 figures