English
Related papers

Related papers: Label Shift Estimators for Non-Ignorable Missing D…

200 papers

Large-scale multi-label classification datasets are commonly, and perhaps inevitably, partially annotated. That is, only a small subset of labels are annotated per sample. Different methods for handling the missing labels induce different…

Computer Vision and Pattern Recognition · Computer Science 2021-10-22 Emanuel Ben-Baruch , Tal Ridnik , Itamar Friedman , Avi Ben-Cohen , Nadav Zamir , Asaf Noy , Lihi Zelnik-Manor

Causal inference with observational data can be performed under an assumption of no unobserved confounders (unconfoundedness assumption). There is, however, seldom clear subject-matter or empirical evidence for such an assumption. We…

Methodology · Statistics 2023-11-13 Minna Genbäck , Xavier de Luna

We study the domain adaptation problem with label shift in this work. Under the label shift context, the marginal distribution of the label varies across the training and testing datasets, while the conditional distribution of features…

Machine Learning · Statistics 2023-05-31 Qinglong Tian , Xin Zhang , Jiwei Zhao

A subvector of predictor that satisfies the ignorability assumption, whose index set is called a sufficient adjustment set, is crucial for conducting reliable causal inference based on observational data. In this paper, we propose a general…

Methodology · Statistics 2024-08-20 Wei Luo , Fei Qin , Lixing Zhu

Data analyses typically rely upon assumptions about missingness mechanisms that lead to observed versus missing data. When the data are missing not at random, direct assumptions about the missingness mechanism, and indirect assumptions…

Methodology · Statistics 2016-03-22 Alexander M Franks , Edoardo M Airoldi , Donald B Rubin

Doubly robust estimators of causal effects are a popular means of estimating causal effects. Such estimators combine an estimate of the conditional mean of the outcome given treatment and confounders (the so-called outcome regression) with…

Methodology · Statistics 2019-01-17 David Benkeser , Weixin Cai , Mark J van der Laan

Off-Policy Evaluation (OPE) aims to estimate the value of a target policy using offline data collected from potentially different policies. In real-world applications, however, logged data often suffers from missingness. While OPE has been…

Machine Learning · Statistics 2025-07-10 Han Wang , Yang Xu , Wenbin Lu , Rui Song

We consider identification and estimation with an outcome missing not at random (MNAR). We study an identification strategy based on a so-called shadow variable. A shadow variable is assumed to be correlated with the outcome, but…

Methodology · Statistics 2019-09-10 Wang Miao , Lan Liu , Eric Tchetgen Tchetgen , Zhi Geng

Multivariate bounded discrete data arises in many fields. In the setting of dementia studies, such data is collected when individuals complete neuropsychological tests. We outline a modeling and inference procedure that can model the joint…

Methodology · Statistics 2026-02-10 Daniel Suen , Yen-Chi Chen

Quantile regression is a powerful tool for detecting exposure-outcome associations given covariates across different parts of the outcome's distribution, but has two major limitations when the aim is to infer the effect of an exposure.…

Missing data is a systemic problem in practical scenarios that causes noise and bias when estimating treatment effects. This makes treatment effect estimation from data with missingness a particularly tricky endeavour. A key reason for this…

Machine Learning · Statistics 2023-02-27 Jeroen Berrevoets , Fergus Imrie , Trent Kyono , James Jordon , Mihaela van der Schaar

Is it possible to perform linear regression on datasets whose labels are shuffled with respect to the inputs? We explore this question by proposing several estimators that recover the weights of a noisy linear model from labels that are…

Machine Learning · Statistics 2017-05-05 Abubakar Abid , Ada Poon , James Zou

Analysis of high-dimensional data, where the number of covariates is larger than the sample size, is a topic of current interest. In such settings, an important goal is to estimate the signal level $\tau^2$ and noise level $\sigma^2$, i.e.,…

Statistics Theory · Mathematics 2023-07-26 Ilan Livne

The purpose of this paper is to provide guidelines for empirical researchers who use a class of bivariate threshold crossing models with dummy endogenous variables. A common practice employed by the researchers is the specification of the…

Econometrics · Economics 2019-03-06 Sukjin Han , Sungwon Lee

In the context of regressing a response $Y$ on a predictor $X$, we consider estimating the local modes of the distribution of $Y$ given $X=x$ when $X$ is prone to measurement error. We propose two nonparametric estimation methods, with one…

Methodology · Statistics 2016-10-28 Haiming Zhou , Xianzheng Huang

Exposure measurement error is a ubiquitous but often overlooked challenge in causal inference with observational data. Existing methods accounting for exposure measurement error largely rely on restrictive parametric assumptions, while…

Despite their impressive performance on a wide variety of tasks, modern language models remain susceptible to distribution shifts, exhibiting brittle behavior when evaluated on data that differs in distribution from their training data. In…

Machine Learning · Computer Science 2026-02-20 Victoria Lin , Louis-Philippe Morency , Eli Ben-Michael

An important challenge in statistical analysis concerns the control of the finite sample bias of estimators. For example, the maximum likelihood estimator has a bias that can result in a significant inferential loss. This problem is…

Statistics Theory · Mathematics 2019-11-04 Stéphane Guerrier , Mucyo Karemera , Samuel Orso , Maria-Pia Victoria-Feser

In this paper, I study the nonparametric identification and estimation of the marginal effect of an endogenous variable $X$ on the outcome variable $Y$, given a potentially mismeasured instrument variable $W^*$, without assuming linearity…

Econometrics · Economics 2024-04-23 Shaomin Wu

How to deal with nonignorable response is often a challenging problem encountered in statistical analysis with missing data. Parametric model assumption for the response mechanism is often made and there is no way to validate the model…

Methodology · Statistics 2018-10-31 Masatoshi Uehara , Jae Kwang Kim