English
Related papers

Related papers: Starting Off on the Wrong Foot: Pitfalls in Data P…

200 papers

In computational materials science, mechanical properties are typically extracted from simulations by means of analysis routines that seek to mimic their experimental counterparts. However, simulated data often exhibit uncertainties that…

Data Analysis, Statistics and Probability · Physics 2017-12-07 Paul N. Patrone , Anthony J. Kearsley , Andrew M. Dienstfrey

In order to fully utilize "big data", it is often required to use "big models". Such models tend to grow with the complexity and size of the training data, and do not make strong parametric assumptions upfront on the nature of the…

Machine Learning · Statistics 2015-04-17 Vikas Sindhwani , Haim Avron

The vast majority of theoretical results in machine learning and statistics assume that the available training data is a reasonably reliable reflection of the phenomena to be learned or estimated. Similarly, the majority of machine learning…

Machine Learning · Computer Science 2017-06-13 Moses Charikar , Jacob Steinhardt , Gregory Valiant

Deep generative frameworks including GANs and normalizing flow models have proven successful at filling in missing values in partially observed data samples by effectively learning -- either explicitly or implicitly -- complex,…

Machine Learning · Computer Science 2021-04-06 Edgar A. Bernal

We present statistical methods for big data arising from online analytical processing, where large amounts of data arrive in streams and require fast analysis without storage/access to the historical data. In particular, we develop…

Computation · Statistics 2018-06-13 Elizabeth D. Schifano , Jing Wu , Chun Wang , Jun Yan , Ming-Hui Chen

Even though a train/test split of the dataset randomly performed is a common practice, could not always be the best approach for estimating performance generalization under some scenarios. The fact is that the usual machine learning…

Machine Learning · Computer Science 2022-09-09 Carlos Catania , Jorge Guerra , Juan Manuel Romero , Gabriel Caffaratti , Martin Marchetta

Benchmarking anomaly detection approaches for multivariate time series is a challenging task due to a lack of high-quality datasets. Current publicly available datasets are too small, not diverse and feature trivial anomalies, which hinders…

Machine Learning · Computer Science 2025-11-13 Lucas Correia , Jan-Christoph Goos , Thomas Bäck , Anna V. Kononova

Deep learning models are widely used across computer vision and other domains. When working on the model induction, selecting the right architecture for a given dataset often relies on repetitive trial-and-error procedures. This procedure…

Machine Learning · Computer Science 2026-01-06 Yen-Chia Chen , Hsing-Kuo Pao , Hanjuan Huang

Many current applications in data science need rich model classes to adequately represent the statistics that may be driving the observations. But rich model classes may be too complex to admit estimators that converge to the truth with…

Information Theory · Computer Science 2022-05-04 N. Santhanam , V. Anantharam , W. Szpankowski

Precise estimation of Crash Modification Factors (CMFs) is central to evaluating the effectiveness of various road safety treatments and prioritizing infrastructure investment accordingly. While customized study for each countermeasure…

Machine Learning · Computer Science 2023-11-16 Yanlin Qi , Jia Li , Michael Zhang

This paper proposes a data and Machine Learning-based forecasting solution for the Telecommunications network-rollout planning problem. Milestone completion-time estimation is crucial to network-rollout planning; accurate estimates enable…

Machine Learning · Computer Science 2022-12-01 Venkatachalam Natchiappan , Shrihari Vasudevan , Thalanayar Muthukumar

This article describes techniques employed in the production of a synthetic dataset of driver telematics emulated from a similar real insurance dataset. The synthetic dataset generated has 100,000 policies that included observations about…

Machine Learning · Statistics 2021-02-02 Banghee So , Jean-Philippe Boucher , Emiliano A. Valdez

Many real world data mining applications involve obtaining predictive models using data sets with strongly imbalanced distributions of the target variable. Frequently, the least common values of this target variable are associated with…

Machine Learning · Computer Science 2015-05-14 Paula Branco , Luis Torgo , Rita Ribeiro

As the use of machine learning in high impact domains becomes widespread, the importance of evaluating safety has increased. An important aspect of this is evaluating how robust a model is to changes in setting or population, which…

Machine Learning · Computer Science 2021-03-16 Adarsh Subbaswamy , Roy Adams , Suchi Saria

The vast advances in Machine Learning over the last ten years have been powered by the availability of suitably prepared data for training purposes. The future of ML-enabled enterprise hinges on data. As such, there is already a vibrant…

Databases · Computer Science 2021-06-02 Yifan Li , Xiaohui Yu , Nick Koudas

Safely deploying machine learning models to the real world is often a challenging process. Models trained with data obtained from a specific geographic location tend to fail when queried with data obtained elsewhere, agents trained in a…

Machine Learning · Computer Science 2021-11-02 Marco Federici , Ryota Tomioka , Patrick Forré

Traffic data imputation is a critical preprocessing step in intelligent transportation systems, underpinning the reliability of downstream transportation services. Despite substantial progress in imputation models, model selection and…

Machine Learning · Computer Science 2025-10-21 Shengnan Guo , Tonglong Wei , Yiheng Huang , Yan Lin , Zekai Shen , Yujuan Dong , Junliang Lin , Youfang Lin , Huaiyu Wan

We propose a novel approach for loss reserving based on deep neural networks. The approach allows for joint modeling of paid losses and claims outstanding, and incorporation of heterogeneous inputs. We validate the models on loss reserving…

Applications · Statistics 2019-09-17 Kevin Kuo

Existing approaches of prescriptive analytics -- where inputs of an optimization model can be predicted by leveraging covariates in a machine learning model -- often attempt to optimize the mean value of an uncertain objective. However,…

Machine Learning · Computer Science 2025-03-05 Dimitris Bertsimas , Benjamin Boucher

Missing data imputation, where a model is trained on observed data to estimate unobserved values, is a fundamental problem in machine learning. In this paper, we rigorously formulate imputation model learning as a mean-squared error risk…

Machine Learning · Statistics 2026-05-14 Luke Shannon , Song Liu , Katarzyna Reluga