Estimating mutual information and multi--information in large networks
Information Theory
2007-07-13 v1 Artificial Intelligence
Computer Vision and Pattern Recognition
Machine Learning
math.IT
Abstract
We address the practical problems of estimating the information relations that characterize large networks. Building on methods developed for analysis of the neural code, we show that reliable estimates of mutual information can be obtained with manageable computational effort. The same methods allow estimation of higher order, multi--information terms. These ideas are illustrated by analyses of gene expression, financial markets, and consumer preferences. In each case, information theoretic measures correlate with independent, intuitive measures of the underlying structures in the system.
Cite
@article{arxiv.cs/0502017,
title = {Estimating mutual information and multi--information in large networks},
author = {Noam Slonim and Gurinder S. Atwal and Gasper Tkacik and William Bialek},
journal= {arXiv preprint arXiv:cs/0502017},
year = {2007}
}