English
Related papers

Related papers: When is $Y_{obs}$ missing and $Y_{mis}$ observed?

200 papers

Consider the regression problem where the response $Y\in\mathbb{R}$ and the covariate $X\in\mathbb{R}^d$ for $d\geq 1$ are \textit{unmatched}. Under this scenario, we do not have access to pairs of observations from the distribution of $(X,…

Statistics Theory · Mathematics 2023-09-19 Mona Azadkia , Fadoua Balabdaoui

The statistical matching problem is a data integration problem with structured missing data. The general form involves the analysis of multiple datasets that only have a strict subset of variables jointly observed across all datasets. The…

Methodology · Statistics 2019-04-01 Daniel Ahfock , Saumyadipta Pyne , Geoffrey J. McLachlan

Complex systems are found in most branches of science. It is still argued how to best quantify their complexity and to what end. One prominent measure of complexity (the statistical complexity) has an operational meaning in terms of the…

Data Analysis, Statistics and Probability · Physics 2011-10-24 Karoline Wiesner , Mile Gu , Elisabeth Rieper , Vlatko Vedral

Missing data is a crucial issue when applying machine learning algorithms to real-world datasets. Starting from the simple assumption that two batches extracted randomly from the same dataset should share the same distribution, we leverage…

Machine Learning · Statistics 2020-07-02 Boris Muzellec , Julie Josse , Claire Boyer , Marco Cuturi

The processing of data which contain missing values is a complicated and always awkward problem, when the data come from real-world contexts. In applications, we are very often in front of observations for which all the values are not…

Statistics Theory · Mathematics 2007-10-04 Marie Cottrell , Patrick Letrémy

We investigate methods for penalized regression in the presence of missing observations. This paper introduces a method for estimating the parameters which compensates for the missing observations. We first, derive an unbiased estimator of…

Applications · Statistics 2013-10-09 Yunjin Choi , Robert Tibshirani

The problem of statistical learning is to construct a predictor of a random variable $Y$ as a function of a related random variable $X$ on the basis of an i.i.d. training sample from the joint distribution of $(X,Y)$. Allowable predictors…

Information Theory · Computer Science 2016-11-15 Maxim Raginsky

The well-known two-slit interference is understood as a special relation between observable (localization at the slits) and state (being on both slits). Relation between an observable and a quantum state is investigated in the general case.…

Quantum Physics · Physics 2009-11-11 Fedor Herbut

In contrast to dyadic interactions, higher-order interactions may contain one another, with subgroups naturally embedded within larger groups. These containment patterns arise empirically in ecology, sociology, computer science and the…

We present a simple yet novel time series imputation technique with the goal of constructing an irregular time series that is uniform across every sample in a data set. Specifically, we fix a grid defined by the midpoints of non-overlapping…

Machine Learning · Computer Science 2022-01-19 Andrew Baumgartner , Sevda Molani , Qi Wei , Jennifer Hadlock

Widely used methods for analyzing missing data can be biased in small samples. To understand these biases, we evaluate in detail the situation where a small univariate normal sample, with values missing at random, is analyzed using either…

Statistics Theory · Mathematics 2017-03-27 Paul T. von Hippel

Score matching is a vital tool for learning the distribution of data with applications across many areas including diffusion processes, energy based modelling, and graphical model estimation. Despite all these applications, little work…

Machine Learning · Statistics 2025-06-03 Josh Givens , Song Liu , Henry W J Reeve

The central challenges in missing data models concern the identifiability of two distributions: the target law and the full law. The target law refers to the joint distribution of the data variables, whereas the full law refers to the joint…

Statistics Theory · Mathematics 2025-10-30 Juha Karvanen , Santtu Tikka

The observed architecture of ecological and socio-economic networks differs significantly from that of random networks. From a network science standpoint, non-random structural patterns observed in real networks call for an explanation of…

Physics and Society · Physics 2019-05-30 Manuel Sebastian Mariani , Zhuo-Ming Ren , Jordi Bascompte , Claudio Juan Tessone

Information-theoretic measures have been widely adopted in the design of features for learning and decision problems. Inspired by this, we look at the relationship between i) a weak form of information loss in the Shannon sense and ii) the…

Machine Learning · Computer Science 2022-01-03 Jorge F. Silva , Felipe Tobar , Mario Vicuña , Felipe Cordova

We consider the problem of estimating the mean of a random variable Y subject to non-ignorable missingness, i.e., where the missingness mechanism depends on Y . We connect the auxiliary proxy variable framework for non-ignorable missingness…

Methodology · Statistics 2023-10-30 Andrew C. Miller , Joseph Futoma

In some multivariate problems with missing data, pairs of variables exist that are never observed together. For example, some modern biological tools can produce data of this form. As a result of this structure, the covariance matrix is…

Methodology · Statistics 2013-08-13 Max Grazier G'Sell , Shai S. Shen-Orr , Robert Tibshirani

Standard methods for anomaly detection assume that all features are observed at both learning time and prediction time. Such methods cannot process data containing missing values. This paper studies five strategies for handling missing…

Machine Learning · Computer Science 2018-09-06 Thomas G. Dietterich , Tadesse Zemicheal

How to learn a good predictor on data with missing values? Most efforts focus on first imputing as well as possible and second learning on the completed data to predict the outcome. Yet, this widespread practice has no theoretical…

Machine Learning · Statistics 2021-12-01 Marine Le Morvan , Julie Josse , Erwan Scornet , Gaël Varoquaux

Data consisting of a graph with a function mapping into $\mathbb{R}^d$ arise in many data applications, encompassing structures such as Reeb graphs, geometric graphs, and knot embeddings. As such, the ability to compare and cluster such…

Computational Geometry · Computer Science 2025-07-17 Erin W. Chambers , Elizabeth Munch , Sarah Percival , Bei Wang