Probabilistic Conformal Coverage Guarantees in Small-Data Settings
Abstract
Conformal prediction provides distribution-free prediction sets with guaranteed marginal coverage. However, in split conformal prediction this guarantee is training-conditional only in expectation: across many calibration draws, the average coverage equals the nominal level, but the realized coverage for a single calibration set may vary substantially. This variance undermines effective risk control in practical applications. Here we introduce the Small Sample Beta Correction (SSBC), a plug-and-play adjustment to the conformal significance level that leverages the exact finite-sample distribution of conformal coverage to provide probabilistic guarantees, ensuring that with user-defined probability over the calibration draw, the deployed predictor achieves at least the desired coverage.
Cite
@article{arxiv.2509.15349,
title = {Probabilistic Conformal Coverage Guarantees in Small-Data Settings},
author = {Petrus H. Zwart},
journal= {arXiv preprint arXiv:2509.15349},
year = {2025}
}