Formal Limitations on the Measurement of Mutual Information
Information Theory
2020-05-21 v4 Machine Learning
math.IT
Machine Learning
Abstract
Measuring mutual information from finite data is difficult. Recent work has considered variational methods maximizing a lower bound. In this paper, we prove that serious statistical limitations are inherent to any method of measuring mutual information. More specifically, we show that any distribution-free high-confidence lower bound on mutual information estimated from N samples cannot be larger than O(ln N ).
Keywords
Cite
@article{arxiv.1811.04251,
title = {Formal Limitations on the Measurement of Mutual Information},
author = {David McAllester and Karl Stratos},
journal= {arXiv preprint arXiv:1811.04251},
year = {2020}
}