English
Related papers

Related papers: Missing Data Imputation for Galaxy Redshift Estima…

200 papers

Given multiband photometric data from the SDSS DR6, we estimate galaxy redshifts. We employ a Random Forest trained on color features and spectroscopic redshifts from 80,000 randomly chosen primary galaxies yielding a mapping from color to…

A common problem faced by statistical institutes is that data may be missing from collected data sets. The typical way to overcome this problem is to impute the missing data. The problem of imputing missing data is complicated by the fact…

Applications · Statistics 2014-01-09 Jeroen Pannekoek , Natalie Shlomo , Ton De Waal

Missing data is a common problem in real-world settings and particularly relevant in healthcare applications where researchers use Electronic Health Records (EHR) and results of observational studies to apply analytics methods. This issue…

Machine Learning · Statistics 2018-12-04 Dimitris Bertsimas , Agni Orfanoudaki , Colin Pawlowski

Flagship near-future surveys targeting $10^8-10^9$ galaxies across cosmic time will soon reveal the processes of galaxy assembly in unprecedented resolution. This creates an immediate computational challenge on effective analyses of the…

Instrumentation and Methods for Astrophysics · Physics 2023-10-03 Bingjie Wang , Joel Leja , V. Ashley Villar , Joshua S. Speagle

This research deals with the estimation and imputation of missing data in longitudinal models with a Poisson response variable inflated with zeros. A methodology is proposed that is based on the use of maximum likelihood, assuming that data…

Methodology · Statistics 2024-09-18 D. S. Martinez-Lobo , O. O. Melo , N. A. Cruz

The number density of galaxy clusters across mass and redshift has been established as a powerful cosmological probe. Cosmological analyses with galaxy clusters traditionally employ scaling relations. However, many challenges arise from…

Cosmology and Nongalactic Astrophysics · Physics 2025-01-08 M. Kosiba , N. Cerardi , M. Pierre , F. Lanusse , C. Garrel , N. Werner , M. Shalak

Incomplete observability of data generates an identification problem. There is no panacea for missing data. What one can learn about a population parameter depends on the assumptions one finds credible to maintain. The credibility of…

Econometrics · Economics 2022-05-17 Charles F. Manski

Accurate and automated galaxy redshift determination is essential for maximizing the scientific return of spectroscopic surveys. In this paper, we propose a data-driven method to address this challenge. The method first learns a rest-frame…

Instrumentation and Methods for Astrophysics · Physics 2026-05-13 Masten Bourahma , Nicolas F. Bouché , Roland Bacon , Johan Richard , Tanya Urrutia , Afonso Vale , Martin Wendt , T. T. Thai

This paper tackles the problem of missing data imputation for noisy and non-Gaussian data. A classical imputation method, the Expectation Maximization (EM) algorithm for Gaussian mixture models, has shown interesting properties when…

Machine Learning · Statistics 2023-05-23 Florian Mouret , Alexandre Hippert-Ferrer , Frédéric Pascal , Jean-Yves Tourneret

The NASA/IPAC Extragalactic Database (NED) is an impressive tool for finding near-exhaustive information on millions of astrophysical objects. Here, we outline a small systematic error that occurs in NED because a low-redshift approximation…

Cosmology and Nongalactic Astrophysics · Physics 2023-01-30 Anthony Carr , Tamara Davis

The problem of missing data, usually absent incurated and competition-standard datasets, is an unfortunate reality for most machine learning models used in industry applications. Recent work has focused on understanding the nature and the…

Machine Learning · Computer Science 2022-01-25 Spyridon Mouselinos , Kyriakos Polymenakos , Antonis Nikitakis , Konstantinos Kyriakopoulos

Photometric redshift estimation is becoming an increasingly important technique, although the currently existing methods present several shortcomings which hinder their application. Here it is shown that most of those drawbacks are…

Astrophysics · Physics 2011-05-05 Narciso Benitez

Widely used methods for analyzing missing data can be biased in small samples. To understand these biases, we evaluate in detail the situation where a small univariate normal sample, with values missing at random, is analyzed using either…

Statistics Theory · Mathematics 2017-03-27 Paul T. von Hippel

Data analysis usually suffers from the Missing Not At Random (MNAR) problem, where the cause of the value missing is not fully observed. Compared to the naive Missing Completely At Random (MCAR) problem, it is more in line with the…

Machine Learning · Computer Science 2025-05-27 Jialei Chen , Yuanbo Xu , Pengyang Wang , Yongjian Yang

We explore the accuracy of the clustering-based redshift inference within the MICE2 simulation. This method uses the spatial clustering of galaxies between a spectroscopic reference sample and an unknown sample. The goal of this study is to…

Cosmology and Nongalactic Astrophysics · Physics 2017-12-27 V. Scottez , A. Benoit-Lévy , J. Coupon , O. Ilbert , Y. Mellier

Imputing missing values is an important preprocessing step in data analysis, but the literature offers little guidance on how to choose between different imputation models. This letter suggests adopting the imputation model that generates a…

Methodology · Statistics 2021-07-13 Moritz Marbach

Missing data are a common problem for both the construction and implementation of a prediction algorithm. Pattern mixture kernel submodels (PMKS) - a series of submodels for every missing data pattern that are fit using only data from that…

Methodology · Statistics 2017-04-27 Sarah Fletcher Mercaldo , Jeffrey D. Blume

The determination of galaxy merger fraction of field galaxies using automatic morphological indices and photometric redshifts is affected by several biases if observational errors are not properly treated. Here, we correct these biases…

Astrophysics · Physics 2009-11-13 C. López-Sanjuan , C. E. García-Dabó , M. Balcells

Cosmology inference of galaxy clustering at the field level with the EFT likelihood in principle allows for extracting all non-Gaussian information from quasi-linear scales, while robustly marginalizing over any astrophysical uncertainties.…

Cosmology and Nongalactic Astrophysics · Physics 2023-10-31 Julia Stadler , Fabian Schmidt , Martin Reinecke

Many physical properties of galaxies correlate with one another, and these correlations are often used to constrain galaxy formation models. Such correlations include the color-magnitude relation, the luminosity-size relation, the…

Astrophysics · Physics 2009-11-13 Graziano Rossi , Ravi K. Sheth