English
Related papers

Related papers: Ignorable and non-ignorable missing data in hidden…

200 papers

Missing data can be informative. Ignoring this information can lead to misleading conclusions when the data model does not allow information to be extracted from the missing data. We propose a co-clustering model, based on the Latent Block…

Machine Learning · Computer Science 2020-10-26 Gabriel Frisch , Jean-Benoist Léger , Yves Grandvalet

When using ecological momentary assessment data (EMA), missing data is pervasive as participant attrition is a common issue. Thus, any EMA study must have a missing data plan. In this paper, we discuss missingness in time series analysis…

Methodology · Statistics 2025-02-18 Lindley R. Slipetz , Ami Falk , Teague R. Henry

Rates of missing data often depend on record-keeping policies and thus may change across times and locations, even when the underlying features are comparatively stable. In this paper, we introduce the problem of Domain Adaptation under…

Machine Learning · Computer Science 2023-05-05 Helen Zhou , Sivaraman Balakrishnan , Zachary C. Lipton

We introduce multiple hidden Markov models (MHMMs) where an observed multivariate categorical time series depends on an unobservable multivariate Mar- kov chain. MHMMs provide an elegant framework for specifying various independence…

Methodology · Statistics 2013-09-17 Roberto Colombi , Sabrina Giordano

We congratulate Nabi et al. (2022) on their impressive and insightful paper, which illustrates the benefits of using causal/counterfactual perspectives and tools in missing data problems. This paper represents an important approach to…

Methodology · Statistics 2025-06-17 Alex W. Levis , Edward H. Kennedy

Current domain adaptation methods under missingness shift are restricted to Missing At Random (MAR) missingness mechanisms. However, in many real-world examples, the MAR assumption may be too restrictive. When covariates are Missing Not At…

Methodology · Statistics 2025-04-02 Tyrel Stokes , Hyungrok Do , Saul Blecker , Rumi Chunara , Samrachana Adhikari

Predicting future operational risk losses gives rise to a significant challenge due to the heterogeneous and time-dependent structures present in real-world data. Furthermore, stress test exercises require examining the relationship with…

Risk Management · Quantitative Finance 2026-04-24 Nikeethan Selvaratnam , Dorinel Bastide , Clément Fernandes , Wojciech Pieczynski

This paper provides further insight into the key concept of missing at random (MAR) in incomplete data analysis. Following the usual selection modelling approach we envisage two models with separable parameters: a model for the response of…

Statistics Theory · Mathematics 2007-06-13 Guobing Lu , John B. Copas

We consider multi-state capture-recapture-recovery data where observed individuals are recorded in a set of possible discrete states. Traditionally, the Arnason-Schwarz model has been fitted to such data where the state process is modeled…

Applications · Statistics 2015-05-20 Ruth King , Roland Langrock

When is keeping a memory of observations worthwhile? We use hidden Markov models to look at phase transitions that emerge when comparing state estimates in systems with discrete states and noisy observations. We infer the underlying state…

Statistical Mechanics · Physics 2017-07-05 Emma Lathouwers , John Bechhoefer

We study a class of missingness mechanisms, called sequentially additive nonignorable, for modeling multivariate data with item nonresponse. These mechanisms explicitly allow the probability of nonresponse for each variable to depend on the…

Methodology · Statistics 2019-02-19 Mauricio Sadinle , Jerome P. Reiter

We introduce the Markov missing graph (MMG), a novel framework that imputes missing data based on undirected graphs. MMG leverages conditional independence relationships to locally decompose the imputation model. To establish the…

Methodology · Statistics 2025-09-04 Yanjiao Yang , Yen-Chi Chen

We illustrate a class of conditional models for the analysis of longitudinal data suffering attrition in random effects models framework, where the subject-specific random effects are assumed to be discrete and to follow a time-dependent…

Methodology · Statistics 2014-04-28 Antonello Maruotti

We consider the filtering of continuous-time finite-state hidden Markov models, where the rate and observation matrices depend on unknown time-dependent parameters, for which no prior or stochastic model is available. We quantify and…

Probability · Mathematics 2021-03-17 Andrew L. Allan

When a missing-data mechanism is NMAR or non-ignorable, missingness is itself vital information and it must be taken into the likelihood, which, however, needs to introduce additional parameters to be estimated. The incompleteness of the…

Methodology · Statistics 2014-05-15 Kosuke Morikawa , Yutaka Kano

Recommender systems are widely used for suggesting books, education materials, and products to users by exploring their behaviors. In reality, users' preferences often change over time, leading to studies on time-dependent recommender…

Information Retrieval · Computer Science 2024-12-17 Haidong Zhang , Wancheng Ni , Xin Li , Yiping Yang

Analysis of competing risks data is often complicated by the incomplete or selectively missing information on the cause of failure. Standard approaches typically assume that the cause of failure is missing at random (MAR), an assumption…

Marginal structural models (MSMs) are commonly used to estimate causal intervention effects in longitudinal non-randomised studies. A common issue when analysing data from observational studies is the presence of incomplete confounder data,…

Methodology · Statistics 2019-12-02 Clemence Leyrat , James R Carpenter , Sebastien Bailly , Elizabeth J Willamson

Hidden semi-Markov models generalise hidden Markov models by explicitly modelling the time spent in a given state, the so-called dwell time, using some distribution defined on the natural numbers. While the (shifted) Poisson and negative…

Methodology · Statistics 2021-02-17 Jennifer Pohle , Timo Adam , Larissa T. Beumer

Progressive diseases worsen over time and are characterised by monotonic change in features that track disease progression. Here we connect ideas from two formerly separate methodologies -- event-based and hidden Markov modelling -- to…

Machine Learning · Computer Science 2021-06-07 Peter A. Wijeratne , Daniel C. Alexander