Statistical Inference for Differentially Private Stochastic Gradient Descent
Abstract
Privacy preservation in machine learning, particularly through Differentially Private Stochastic Gradient Descent (DP-SGD), is critical for sensitive data analysis. However, existing statistical inference methods for SGD predominantly focus on cyclic subsampling, while DP-SGD requires randomized subsampling. This paper first bridges this gap by establishing the asymptotic properties of SGD under the randomized rule and extending these results to DP-SGD. For the output of DP-SGD, we show that the asymptotic variance decomposes into statistical, sampling, and privacy-induced components. Two methods are proposed for constructing valid confidence intervals: the plug-in method and the random scaling method. We also perform extensive numerical analysis, which shows that the proposed confidence intervals achieve nominal coverage rates while maintaining privacy.
Cite
@article{arxiv.2507.20560,
title = {Statistical Inference for Differentially Private Stochastic Gradient Descent},
author = {Xintao Xia and Linjun Zhang and Zhanrui Cai},
journal= {arXiv preprint arXiv:2507.20560},
year = {2025}
}