Eliminating Multicollinearity Issues in Neural Network Ensembles: Incremental, Negatively Correlated, Optimal Convex Blending
Machine Learning
2021-05-03 v1 Artificial Intelligence
Optimization and Control
Abstract
Given a {features, target} dataset, we introduce an incremental algorithm that constructs an aggregate regressor, using an ensemble of neural networks. It is well known that ensemble methods suffer from the multicollinearity issue, which is the manifestation of redundancy arising mainly due to the common training-dataset. In the present incremental approach, at each stage we optimally blend the aggregate regressor with a newly trained neural network under a convexity constraint which, if necessary, induces negative correlations. Under this framework, collinearity issues do not arise at all, rendering so the method both accurate and robust.
Keywords
Cite
@article{arxiv.2104.14715,
title = {Eliminating Multicollinearity Issues in Neural Network Ensembles: Incremental, Negatively Correlated, Optimal Convex Blending},
author = {Pola Lydia Lagari and Lefteri H. Tsoukalas and Salar Safarkhani and Isaac E. Lagaris},
journal= {arXiv preprint arXiv:2104.14715},
year = {2021}
}