Expectation Propagation for Approximate Inference: Free Probability Framework
Abstract
We study asymptotic properties of expectation propagation (EP) -- a method for approximate inference originally developed in the field of machine learning. Applied to generalized linear models, EP iteratively computes a multivariate Gaussian approximation to the exact posterior distribution. The computational complexity of the repeated update of covariance matrices severely limits the application of EP to large problem sizes. In this study, we present a rigorous analysis by means of free probability theory that allows us to overcome this computational bottleneck if specific data matrices in the problem fulfill certain properties of asymptotic freeness. We demonstrate the relevance of our approach on the gene selection problem of a microarray dataset.
Cite
@article{arxiv.1801.05411,
title = {Expectation Propagation for Approximate Inference: Free Probability Framework},
author = {Burak Çakmak and Manfred Opper},
journal= {arXiv preprint arXiv:1801.05411},
year = {2018}
}
Comments
Both authors are co-first authors. The main body of this paper is accepted for publication in the proceedings of the 2018 IEEE International Symposium on Information Theory (ISIT)