Entropy and inference, revisited
数据分析、统计与概率
2007-05-23 v2
摘要
We study properties of popular near-uniform (Dirichlet) priors for learning undersampled probability distributions on discrete nonmetric spaces and show that they lead to disastrous results. However, an Occam-style phase space argument expands the priors into their infinite mixture and resolves most of the observed problems. This leads to a surprisingly good estimator of entropies of discrete distributions.
引用
@article{arxiv.physics/0108025,
title = {Entropy and inference, revisited},
author = {Ilya Nemenman and Fariel Shafee and William Bialek},
journal= {arXiv preprint arXiv:physics/0108025},
year = {2007}
}
备注
LaTex2e, 9 pages, 5 figures; references added, minor revisions introduced, formatting errors corrected