Auditing ML Models for Individual Bias and Unfairness
Machine Learning
2020-03-12 v1 Machine Learning
Abstract
We consider the task of auditing ML models for individual bias/unfairness. We formalize the task in an optimization problem and develop a suite of inferential tools for the optimal value. Our tools permit us to obtain asymptotic confidence intervals and hypothesis tests that cover the target/control the Type I error rate exactly. To demonstrate the utility of our tools, we use them to reveal the gender and racial biases in Northpointe's COMPAS recidivism prediction instrument.
Cite
@article{arxiv.2003.05048,
title = {Auditing ML Models for Individual Bias and Unfairness},
author = {Songkai Xue and Mikhail Yurochkin and Yuekai Sun},
journal= {arXiv preprint arXiv:2003.05048},
year = {2020}
}
Comments
In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS) 2020