Expectation-maximization for logistic regression
Computation
2013-06-04 v1 Statistics Theory
Machine Learning
Statistics Theory
Abstract
We present a family of expectation-maximization (EM) algorithms for binary and negative-binomial logistic regression, drawing a sharp connection with the variational-Bayes algorithm of Jaakkola and Jordan (2000). Indeed, our results allow a version of this variational-Bayes approach to be re-interpreted as a true EM algorithm. We study several interesting features of the algorithm, and of this previously unrecognized connection with variational Bayes. We also generalize the approach to sparsity-promoting priors, and to an online method whose convergence properties are easily established. This latter method compares favorably with stochastic-gradient descent in situations with marked collinearity.
Cite
@article{arxiv.1306.0040,
title = {Expectation-maximization for logistic regression},
author = {James G. Scott and Liang Sun},
journal= {arXiv preprint arXiv:1306.0040},
year = {2013}
}