IT formulae for gamma target: mutual information and relative entropy
Abstract
In this paper, we introduce new Stein identities for gamma target distribution as well as a new non-linear channel specifically designed for gamma inputs. From these two ingredients, we derive an explicit and simple formula for the derivative of the input-output mutual information of this non-linear channel with respect to the channel quality parameter. This relation is reminiscent of the well-known link between the derivative of the input-output mutual information of additive Gaussian noise channel with respect to the signal-to-noise ratio and the minimum mean-square error. The proof relies on a rescaled version of De Bruijn identity for gamma target distribution together with a stochastic representation for the gamma specific Fisher information. Finally, we are able to derive precise bounds and asymptotics for the input-output mutual information of the non-linear channel with gamma inputs.
Keywords
Cite
@article{arxiv.1609.02342,
title = {IT formulae for gamma target: mutual information and relative entropy},
author = {Benjamin Arras and Yvik Swan},
journal= {arXiv preprint arXiv:1609.02342},
year = {2019}
}
Comments
submitted for publication in the IEEE Transactions on Information Theory