Informative Features for Model Comparison
Machine Learning
2018-10-30 v1 Machine Learning
Abstract
Given two candidate models, and a set of target observations, we address the problem of measuring the relative goodness of fit of the two models. We propose two new statistical tests which are nonparametric, computationally efficient (runtime complexity is linear in the sample size), and interpretable. As a unique advantage, our tests can produce a set of examples (informative features) indicating the regions in the data domain where one model fits significantly better than the other. In a real-world problem of comparing GAN models, the test power of our new test matches that of the state-of-the-art test of relative goodness of fit, while being one order of magnitude faster.
Cite
@article{arxiv.1810.11630,
title = {Informative Features for Model Comparison},
author = {Wittawat Jitkrittum and Heishiro Kanagawa and Patsorn Sangkloy and James Hays and Bernhard Schölkopf and Arthur Gretton},
journal= {arXiv preprint arXiv:1810.11630},
year = {2018}
}
Comments
Accepted to NIPS 2018