English
Related papers

Related papers: Ignorable and non-ignorable missing data in hidden…

200 papers

Practical problems with missing data are common, and statistical methods have been developed concerning the validity and/or efficiency of statistical procedures. On a central focus, there have been longstanding interests on the mechanism…

Methodology · Statistics 2020-03-26 Rui Duan , C. Jason Liang , Pamela Shaw , Cheng Yong Tang , Yong Chen

We present a new method for inferring hidden Markov models from noisy time sequences without the necessity of assuming a model architecture, thus allowing for the detection of degenerate states. This is based on the statistical prediction…

Quantitative Methods · Quantitative Biology 2012-01-24 David Kelly , Mark Dillingham , Andrew Hudson , Karoline Wiesner

Health economic evaluations based on patient-level data collected alongside clinical trials~(e.g. health related quality of life and resource use measures) are an important component of the process which informs resource allocation…

Applications · Statistics 2020-05-25 Andrea Gabrio , Rachael Hunter , Alexina J. Mason , Gianluca Baio

During the past few decades, missing-data problems have been studied extensively, with a focus on the ignorable missing case, where the missing probability depends only on observable quantities. By contrast, research into non-ignorable…

Methodology · Statistics 2019-08-06 Yukun Liu , Pengfei Li , Jing Qin

Nonignorable missing data, where the probability of missingness depends on unobserved values, presents a significant challenge in statistical analysis. Traditional methods often rely on strong parametric assumptions that are difficult to…

Methodology · Statistics 2025-09-19 Yujie Zhao

In Fernandez-Fontelo et al (Statis. Med. 2016, DOI 10.1002/sim.7026) hidden integer-valued autoregressive (INAR) processes are used to estimate reporting probabilities for various diseases. In this comment it is demonstrated that the…

Methodology · Statistics 2019-03-01 Johannes Bracher

Constant (naive) imputation is still widely used in practice as this is a first easy-to-use technique to deal with missing data. Yet, this simple method could be expected to induce a large bias for prediction purposes, as the imputed input…

Statistics Theory · Mathematics 2024-02-07 Alexis Ayme , Claire Boyer , Aymeric Dieuleveut , Erwan Scornet

Data analysis often encounters missing data, which can result in inaccurate conclusions, especially when it comes to ordinal variables. In trauma data, the Glasgow Coma Scale is useful for assessing the level of consciousness. This score is…

Methodology · Statistics 2025-07-01 Abdoulaye Dioni , Alexandre Bureau , Lynne Moore , Aida Eslami

Multistate Markov models are a canonical parametric approach for data modeling of observed or latent stochastic processes supported on a finite state space. Continuous-time Markov processes describe data that are observed irregularly over…

Continuous-time multistate models are widely used for analyzing interval-censored data on disease progression over time. Sometimes, diseases manifest differently and what appears to be a coherent collection of symptoms is the expression of…

Methodology · Statistics 2024-10-08 Yidan Shi , Leilei Zeng , Mary E. Thompson , Suzanne L. Tyas

Time series and sequential data have gained significant attention recently since many real-world processes in various domains such as finance, education, biology, and engineering can be modeled as time series. Although many algorithms and…

Machine Learning · Computer Science 2020-08-11 Manie Tadayon , Greg Pottie

Hidden Markov models are versatile tools for modeling sequential observations, where it is assumed that a hidden state process selects which of finitely many distributions generates any given observation. Specifically for time series of…

Methodology · Statistics 2019-01-11 Timo Adam , Roland Langrock , Christian H. Weiß

Missing Not at Random (MNAR) and nonnormal data are challenging to handle. Traditional missing data analytical techniques such as full information maximum likelihood estimation (FIML) may fail with nonnormal data as they are built on normal…

Applications · Statistics 2024-06-21 Dandan Tang , Xin Tong

The well-established methodology for the estimation of hidden semi-Markov models (HSMMs) as hidden Markov models (HMMs) with extended state spaces is further developed to incorporate covariate influences across all aspects of the state…

Methodology · Statistics 2024-05-24 Jan-Ole Koslik

Missing data frequently arises across diverse domains, including time-series and image domains. In the real world, missing occurrences often depend on the unobservable values themselves, which are referred to as Missing Not at Random…

Machine Learning · Computer Science 2026-05-26 Gyuwon Sim , Sumin Lee , Heesun Bae , Byeonghu Na , Doyun Kwon , Ju-Hee Hwang , Jae-Young Lim , Il-Chul Moon

Hidden Markov models (HMMs) are flexible time series models in which the distributions of the observations depend on unobserved serially correlated states. The state-dependent distributions in HMMs are usually taken from some class of…

Methodology · Statistics 2014-06-19 Roland Langrock , Thomas Kneib , Alexander Sohn , Stacy DeRuiter

Hidden Markov Models (HMMs) comprise a powerful generative approach for modeling sequential data and time-series in general. However, the commonly employed assumption of the dependence of the current time frame to a single or multiple…

Machine Learning · Computer Science 2021-09-13 Konstantinos P. Panousis , Sotirios Chatzis , Sergios Theodoridis

Two major ideas in the analysis of missing data are (a) the EM algorithm [Dempster, Laird and Rubin, J. Roy. Statist. Soc. Ser. B 39 (1977) 1--38] for maximum likelihood (ML) estimation, and (b) the formulation of models for the joint…

Methodology · Statistics 2011-04-14 Yan Zhou , Roderick J. A. Little , John D. Kalbfleisch

Missing data imputation, where a model is trained on observed data to estimate unobserved values, is a fundamental problem in machine learning. In this paper, we rigorously formulate imputation model learning as a mean-squared error risk…

Machine Learning · Statistics 2026-05-14 Luke Shannon , Song Liu , Katarzyna Reluga

The analysis of high-dimensional time series data has become increasingly important across a wide range of fields. Recently, a method for constructing the minimum information Markov kernel on finite state spaces was established. In this…

Methodology · Statistics 2026-01-13 Issey Sukeda , Tomonari Sei