A Non-Parametric Test to Detect Data-Copying in Generative Models
Machine Learning
2020-04-14 v1 Machine Learning
Abstract
Detecting overfitting in generative models is an important challenge in machine learning. In this work, we formalize a form of overfitting that we call {\em{data-copying}} -- where the generative model memorizes and outputs training samples or small variations thereof. We provide a three sample non-parametric test for detecting data-copying that uses the training set, a separate sample from the target distribution, and a generated sample from the model, and study the performance of our test on several canonical models and datasets. For code \& examples, visit https://github.com/casey-meehan/data-copying
Keywords
Cite
@article{arxiv.2004.05675,
title = {A Non-Parametric Test to Detect Data-Copying in Generative Models},
author = {Casey Meehan and Kamalika Chaudhuri and Sanjoy Dasgupta},
journal= {arXiv preprint arXiv:2004.05675},
year = {2020}
}
Comments
To be published in AISTATS 2020