Dealing with missing data using attention and latent space regularization
Abstract
Most practical data science problems encounter missing data. A wide variety of solutions exist, each with strengths and weaknesses that depend upon the missingness-generating process. Here we develop a theoretical framework for training and inference using only observed variables enabling modeling of incomplete datasets without imputation. Using an information and measure-theoretic argument we construct models with latent space representations that regularize against the potential bias introduced by missing data. The theoretical properties of this approach are demonstrated empirically using a synthetic dataset. The performance of this approach is tested on 11 benchmarking datasets with missingness and 18 datasets corrupted across three missingness patterns with comparison against a state-of-the-art model and industry-standard imputation. We show that our proposed method overcomes the weaknesses of imputation methods and outperforms the current state-of-the-art.
Cite
@article{arxiv.2211.07059,
title = {Dealing with missing data using attention and latent space regularization},
author = {Jahan C. Penny-Dimri and Christoph Bergmeir and Julian Smith},
journal= {arXiv preprint arXiv:2211.07059},
year = {2022}
}