Near-Optimal Generalized Private Testing

In differential privacy (DP), the generalized private testing problem was introduced by Liu and Talwar (STOC 2019). Given a dataset $X \in \mathcal{X}$ and a sequence of black-box $\varepsilon_t$-DP mechanisms $M_t:\mathcal{X}\to\{+1,-1\}$, the analyst must accept the first mechanism whose success probability $p_t=\Pr[M_t(X)=+1]$ exceeds a given threshold $p^*\in(0,1)$, while achieving DP. Accuracy is measured by the gap between $p^*$ and a rejection threshold $\bar{p}$, such that with probability $1-\beta$ for all $t\geq1$, if $p_t\leq\bar{p}$, then $M_t$ is rejected, and if $p_t\geq p^*$, then it is accepted. This generalizes the standard private testing problem, whose solution, the Sparse Vector Technique, is ubiquitous in DP. We introduce the Generalized Thresholding Mechanism (GTM) for generalized private testing. For $\varepsilon>0$ and any sequence of $(\varepsilon_t,\delta_t)$-DP mechanisms $M_t$, the GTM is pure $\varepsilon$-DP. For $\theta>0$, $\gamma\in(1,2]$, and $\beta\in(0,1)$, $\bar{p}_t=\max(p^*/\gamma\Lambda_t, 1 - \gamma\Lambda_t(1-p^*))-\delta_t/\varepsilon_t$ for $\Lambda_t=(5t\ln^3(t+2))^{(2+\theta)\varepsilon_t/\varepsilon}(4/\beta)^{(3+\theta+2/\theta)\varepsilon_t/\varepsilon}$. With probability $1-\beta$, the number of evaluations of $M_t$ is at most $O((\ln(t/\beta)/(\gamma-1)^2)\max(\Lambda_t/p^*,(1-p^*)^{-1}))$ for all $t\geq 1$. Our lower bounds prove near-optimality of our accuracy and sample complexity guarantees. Via the GTM, we give a black-box reduction for DP optimization from the continual observation (CO) setting to the batch setting. This gives us the first DP-CO algorithms for many maximization problems. Further, the GTM permits an adaptive choice of acceptance thresholds $(p^*_t)_{t\geq1}$, addressing a challenge mentioned in prior work on using generalized private testing for hyperparameter optimization (Papernot and Steinke (ICLR 2022)).