Estimating the size of a set using cascading exclusion
Abstract
Let be a finite set, and an i.i.d. uniform sample from . To estimate the size , without further structure, one can wait for repeats and use the birthday problem. This requires a sample size of the order . On the other hand, if , the maximum of the sample blown up by gives an efficient estimator based on any growing sample size. This paper gives refinements that interpolate between these extremes. A general non-asymptotic theory is developed. This includes estimating the volume of a compact convex set, the unseen species problem, and a host of testing problems that follow from the question `Is this new observation a typical pick from a large prespecified population?' We also treat regression style predictors. A general theorem gives non-parametric finite error bounds in all cases.
Cite
@article{arxiv.2508.05901,
title = {Estimating the size of a set using cascading exclusion},
author = {Sourav Chatterjee and Persi Diaconis and Susan Holmes},
journal= {arXiv preprint arXiv:2508.05901},
year = {2026}
}
Comments
52 pages, 10 figures. Minor changes in this revision. To appear in Statistical Science