English

Information Bottleneck on General Alphabets

Information Theory 2018-05-02 v2 math.IT

Abstract

We prove rigorously a source coding theorem that can probably be considered folklore, a generalization to arbitrary alphabets of a problem motivated by the Information Bottleneck method. For general random variables (Y,X)(Y, X), we show essentially that for some nNn \in \mathbb{N}, a function ff with rate limit logfnR\log|f| \le nR and I(Yn;f(Xn))nSI(Y^n; f(X^n)) \ge nS exists if and only if there is a random variable UU such that the Markov chain YXUY - X - U holds, I(U;X)RI(U; X) \le R and I(U;Y)SI(U; Y) \ge S. The proof relies on the well established discrete case and showcases a technique for lifting discrete coding theorems to arbitrary alphabets.

Keywords

Cite

@article{arxiv.1801.01050,
  title  = {Information Bottleneck on General Alphabets},
  author = {Georg Pichler and Günther Koliander},
  journal= {arXiv preprint arXiv:1801.01050},
  year   = {2018}
}

Comments

extended version, presented at ISIT 2018, Vail, CO

R2 v1 2026-06-22T23:35:34.261Z