A perturbative approach to non-linearities in the information carried by a two layer neural network
Statistical Mechanics
2009-10-31 v1
Abstract
We evaluate the mutual information between the input and the output of a two layer network in the case of a noisy and non-linear analogue channel. In the case where the non-linearity is small with respect to the variability in the noise, we derive an exact expression for the contribution to the mutual information given by the non-linear term in first order of perturbation theory. Finally we show how the calculation can be simplified by means of a diagrammatic expansion. Our results suggest that the use of perturbation theories applied to neural systems might give an insight on the contribution of non-linearities to the information transmission and in general to the neuronal dynamics.
Keywords
Cite
@article{arxiv.cond-mat/0010249,
title = {A perturbative approach to non-linearities in the information carried by a two layer neural network},
author = {E. Korutcheva and V. Del Prete and J. -P. Nadal},
journal= {arXiv preprint arXiv:cond-mat/0010249},
year = {2009}
}
Comments
Accepted as a preprint of ICTP, Trieste