Fast and More Powerful Selective Inference for Sparse High-order Interaction Model
Abstract
Automated high-stake decision-making such as medical diagnosis requires models with high interpretability and reliability. As one of the interpretable and reliable models with good prediction ability, we consider Sparse High-order Interaction Model (SHIM) in this study. However, finding statistically significant high-order interactions is challenging due to the intrinsic high dimensionality of the combinatorial effects. Another problem in data-driven modeling is the effect of "cherry-picking" a.k.a. selection bias. Our main contribution is to extend the recently developed parametric programming approach for selective inference to high-order interaction models. Exhaustive search over the cherry tree (all possible interactions) can be daunting and impractical even for a small-sized problem. We introduced an efficient pruning strategy and demonstrated the computational efficiency and statistical power of the proposed method using both synthetic and real data.
Cite
@article{arxiv.2106.04929,
title = {Fast and More Powerful Selective Inference for Sparse High-order Interaction Model},
author = {Diptesh Das and Vo Nguyen Le Duy and Hiroyuki Hanada and Koji Tsuda and Ichiro Takeuchi},
journal= {arXiv preprint arXiv:2106.04929},
year = {2021}
}