English
Related papers

Related papers: Prediction with Missing Data: Target Probabilities…

200 papers

We consider missing data in the context of hidden Markov models with a focus on situations where data is missing not at random (MNAR) and missingness depends on the identity of the hidden states. In simulations, we show that including a…

Methodology · Statistics 2021-09-08 Maarten Speekenbrink , Ingmar Visser

The presence of missing values makes supervised learning much more challenging. Indeed, previous work has shown that even when the response is a linear function of the complete data, the optimal predictor is a complex function of the…

Machine Learning · Computer Science 2020-11-05 Marine Le Morvan , Julie Josse , Thomas Moreau , Erwan Scornet , Gaël Varoquaux

Missing data are frequently encountered in various disciplines and can be divided into three categories: missing completely at random (MCAR), missing at random (MAR) and missing not at random (MNAR). Valid statistical approaches to missing…

Methodology · Statistics 2021-05-28 Hairu Wang , Zhiping Lu , Yukun Liu

Practical problems with missing data are common, and statistical methods have been developed concerning the validity and/or efficiency of statistical procedures. On a central focus, there have been longstanding interests on the mechanism…

Methodology · Statistics 2020-03-26 Rui Duan , C. Jason Liang , Pamela Shaw , Cheng Yong Tang , Yong Chen

This paper provides further insight into the key concept of missing at random (MAR) in incomplete data analysis. Following the usual selection modelling approach we envisage two models with separable parameters: a model for the response of…

Statistics Theory · Mathematics 2007-06-13 Guobing Lu , John B. Copas

Handling missing values at test time is challenging for machine learning models, especially when aiming for both high accuracy and interpretability. Established approaches often add bias through imputation or excessive model complexity via…

Machine Learning · Computer Science 2025-05-07 Lena Stempfle , Anton Matsson , Newton Mwai , Fredrik D. Johansson

Real-world datasets often have missing values associated with complex generative processes, where the cause of the missingness may not be fully observed. This is known as missing not at random (MNAR) data. However, many imputation methods…

Machine Learning · Computer Science 2021-10-29 Chao Ma , Cheng Zhang

Training datasets for machine learning often have some form of missingness. For example, to learn a model for deciding whom to give a loan, the available training data includes individuals who were given a loan in the past, but not those…

Machine Learning · Computer Science 2020-12-22 Naman Goel , Alfonso Amayuelas , Amit Deshpande , Amit Sharma

Large-scale traffic forecasting relies on fixed sensor networks that often exhibit blackouts: contiguous intervals of missing measurements caused by detector or communication failures. These outages are typically handled under a Missing At…

Machine Learning · Statistics 2026-01-07 Aman Sunesh , Allan Ma , Siddarth Nilol

Analysis of competing risks data is often complicated by the incomplete or selectively missing information on the cause of failure. Standard approaches typically assume that the cause of failure is missing at random (MAR), an assumption…

Missing data is an universal problem in statistics. We develop a unified framework for estimating parameters defined by general estimating equations under a missing-at-random (MAR) mechanism, based on generalized entropy calibration…

Methodology · Statistics 2026-03-31 Mst Moushumi Pervin , Hengfang Wang , Jae Kwang Kim

In many application settings, the data have missing entries which make analysis challenging. An abundant literature addresses missing values in an inferential framework: estimating parameters and their variance from incomplete tables. Here,…

Machine Learning · Statistics 2024-03-22 Julie Josse , Jacob M. Chen , Nicolas Prost , Erwan Scornet , Gaël Varoquaux

Missing values pose a persistent challenge in modern data science. Consequently, there is an ever-growing number of publications introducing new imputation methods in various fields. The present paper attempts to take a step back and…

Statistics Theory · Mathematics 2026-01-21 Jeffrey Näf , Erwan Scornet , Julie Josse

During the past few decades, missing-data problems have been studied extensively, with a focus on the ignorable missing case, where the missing probability depends only on observable quantities. By contrast, research into non-ignorable…

Methodology · Statistics 2019-08-06 Yukun Liu , Pengfei Li , Jing Qin

Clinical researchers often select among and evaluate risk prediction models using standard machine learning metrics based on confusion matrices. However, if these models are used to allocate interventions to patients, standard metrics…

Machine Learning · Statistics 2020-06-03 Alejandro Schuler , Aashish Bhardwaj , Vincent Liu

Structural missingness breaks 'just impute and train': values can be undefined by causal or logical constraints, and the mask may depend on observed variables, unobserved variables (MNAR), and other missingness indicators. It simultaneously…

Missing data are prevalent and present daunting challenges in real data analysis. While there is a growing body of literature on fairness in analysis of fully observed data, there has been little theoretical work on investigating fairness…

Machine Learning · Computer Science 2021-12-10 Yiliang Zhang , Qi Long

Missing data occur frequently in empirical studies in health and social sciences, often compromising our ability to make accurate inferences. An outcome is said to be missing not at random (MNAR) if, conditional on the observed variables,…

Methodology · Statistics 2019-01-23 BaoLuo Sun , Lan Liu , Wang Miao , Kathleen Wirth , James Robins , Eric Tchetgen Tchetgen

We present a method for incorporating missing data in non-parametric statistical learning without the need for imputation. We focus on a tree-based method, Bayesian Additive Regression Trees (BART), enhanced with "Missingness Incorporated…

Machine Learning · Statistics 2014-02-14 Adam Kapelner , Justin Bleich

Missing data frequently arises across diverse domains, including time-series and image domains. In the real world, missing occurrences often depend on the unobservable values themselves, which are referred to as Missing Not at Random…

Machine Learning · Computer Science 2026-05-26 Gyuwon Sim , Sumin Lee , Heesun Bae , Byeonghu Na , Doyun Kwon , Ju-Hee Hwang , Jae-Young Lim , Il-Chul Moon