Private Prediction Sets
Abstract
In real-world settings involving consequential decision-making, the deployment of machine learning systems generally requires both reliable uncertainty quantification and protection of individuals' privacy. We present a framework that treats these two desiderata jointly. Our framework is based on conformal prediction, a methodology that augments predictive models to return prediction sets that provide uncertainty quantification -- they provably cover the true response with a user-specified probability, such as 90%. One might hope that when used with privately-trained models, conformal prediction would yield privacy guarantees for the resulting prediction sets; unfortunately, this is not the case. To remedy this key problem, we develop a method that takes any pre-trained predictive model and outputs differentially private prediction sets. Our method follows the general approach of split conformal prediction; we use holdout data to calibrate the size of the prediction sets but preserve privacy by using a privatized quantile subroutine. This subroutine compensates for the noise introduced to preserve privacy in order to guarantee correct coverage. We evaluate the method on large-scale computer vision datasets.
Cite
@article{arxiv.2102.06202,
title = {Private Prediction Sets},
author = {Anastasios N. Angelopoulos and Stephen Bates and Tijana Zrnic and Michael I. Jordan},
journal= {arXiv preprint arXiv:2102.06202},
year = {2024}
}
Comments
Code available at https://github.com/aangelopoulos/private_prediction_sets