Over the years, ensemble methods have become a staple of machine learning. Similarly, generalized linear models (GLMs) have become very popular for a wide variety of statistical inference tasks. The former have been shown to enhance out- of-sample predictive power and the latter possess easy interpretability. Recently, ensembles of GLMs have been proposed as a possibility. On the downside, this approach loses the interpretability that GLMs possess. We show that minimum description length (MDL)-motivated compression of the inferred ensembles can be used to recover interpretability without much, if any, downside to performance and illustrate on a number of standard classification data sets.
@article{arxiv.1611.06800,
title = {MDL-motivated compression of GLM ensembles increases interpretability and retains predictive power},
author = {Boris Hayete and Matthew Valko and Alex Greenfield and Raymond Yan},
journal= {arXiv preprint arXiv:1611.06800},
year = {2016}
}
Comments
The authors would like to acknowledge Leon Furchtgott and Fred Gruber for their invaluable feedback on the manuscript, and Fred Gruber for his help with LATEX. Presented at NIPS 2016 Workshop on Interpretable Machine Learning in Complex Systems