Pre-processing with Orthogonal Decompositions for High-dimensional Explanatory Variables
Abstract
Strong correlations between explanatory variables are problematic for high-dimensional regularized regression methods. Due to the violation of the Irrepresentable Condition, the popular LASSO method may suffer from false inclusions of inactive variables. In this paper, we propose pre-processing with orthogonal decompositions (PROD) for the explanatory variables in high-dimensional regressions. The PROD procedure is constructed based upon a generic orthogonal decomposition of the design matrix. We demonstrate by two concrete cases that the PROD approach can be effectively constructed for improving the performance of high-dimensional penalized regression. Our theoretical analysis reveals their properties and benefits for high-dimensional penalized linear regression with LASSO. Extensive numerical studies with simulations and data analysis show the promising performance of the PROD.
Cite
@article{arxiv.2106.09071,
title = {Pre-processing with Orthogonal Decompositions for High-dimensional Explanatory Variables},
author = {Xu Han and Ethan X Fang and Cheng Yong Tang},
journal= {arXiv preprint arXiv:2106.09071},
year = {2021}
}