Automated Dependence Plots
Abstract
In practical applications of machine learning, it is necessary to look beyond standard metrics such as test accuracy in order to validate various qualitative properties of a model. Partial dependence plots (PDP), including instance-specific PDPs (i.e., ICE plots), have been widely used as a visual tool to understand or validate a model. Yet, current PDPs suffer from two main drawbacks: (1) a user must manually sort or select interesting plots, and (2) PDPs are usually limited to plots along a single feature. To address these drawbacks, we formalize a method for automating the selection of interesting PDPs and extend PDPs beyond showing single features to show the model response along arbitrary directions, for example in raw feature space or a latent space arising from some generative model. We demonstrate the usefulness of our automated dependence plots (ADP) across multiple use-cases and datasets including model selection, bias detection, understanding out-of-sample behavior, and exploring the latent space of a generative model.
Cite
@article{arxiv.1912.01108,
title = {Automated Dependence Plots},
author = {David I. Inouye and Liu Leqi and Joon Sik Kim and Bryon Aragam and Pradeep Ravikumar},
journal= {arXiv preprint arXiv:1912.01108},
year = {2020}
}
Comments
In Uncertainty in Artificial Intelligence (UAI 2020). Camera-ready version. Code is available at https://github.com/davidinouye/adp