Statistical Distortion: Consequences of Data Cleaning
Databases
2012-08-10 v1
Abstract
We introduce the notion of statistical distortion as an essential metric for measuring the effectiveness of data cleaning strategies. We use this metric to propose a widely applicable yet scalable experimental framework for evaluating data cleaning strategies along three dimensions: glitch improvement, statistical distortion and cost-related criteria. Existing metrics focus on glitch improvement and cost, but not on the statistical impact of data cleaning strategies. We illustrate our framework on real world data, with a comprehensive suite of experiments and analyses.
Cite
@article{arxiv.1208.1932,
title = {Statistical Distortion: Consequences of Data Cleaning},
author = {Tamraparni Dasu and Ji Meng Loh},
journal= {arXiv preprint arXiv:1208.1932},
year = {2012}
}
Comments
VLDB2012