Maximally Informative Statistics
Data Analysis, Statistics and Probability
2007-05-23 v1
Abstract
In this paper we propose a Bayesian, information theoretic approach to dimensionality reduction. The approach is formulated as a variational principle on mutual information, and seamlessly addresses the notions of sufficiency, relevance, and representation. Maximally informative statistics are shown to minimize a Kullback-Leibler distance between posterior distributions. Illustrating the approach, we derive the maximally informative one dimensional statistic for a random sample from the Cauchy distribution.
Keywords
Cite
@article{arxiv.physics/0010039,
title = {Maximally Informative Statistics},
author = {David R. Wolf and Edward I. George},
journal= {arXiv preprint arXiv:physics/0010039},
year = {2007}
}
Comments
13 pages. Presented Bayesian Statistics 6, Valencia, 1998. Arxiv version asserts bold vectors dropped in print