Information Loss in Static Nonlinearities
Information Theory
2012-02-03 v2 math.IT
Exactly Solvable and Integrable Systems
Abstract
In this work, conditional entropy is used to quantify the information loss induced by passing a continuous random variable through a memoryless nonlinear input-output system. We derive an expression for the information loss depending on the input density and the nonlinearity and show that the result is strongly related to the non-injectivity of the considered system. Tight upper bounds are presented, which can be evaluated with less difficulty than a direct evaluation of the information loss, which involves the logarithm of a sum. Application of our results is illustrated on a set of examples.
Keywords
Cite
@article{arxiv.1102.4794,
title = {Information Loss in Static Nonlinearities},
author = {Bernhard C. Geiger and Christian Feldbauer and Gernot Kubin},
journal= {arXiv preprint arXiv:1102.4794},
year = {2012}
}
Comments
9 pages, 6 figures; A short version of this paper is submitted to an IEEE conference