PilotDB: Database-Agnostic Online Approximate Query Processing with A Priori Error Guarantees (Technical Report)
Abstract
After decades of research in approximate query processing (AQP), its adoption in the industry remains limited. Existing methods struggle to simultaneously provide user-specified error guarantees, eliminate maintenance overheads, and avoid modifications to database management systems. To address these challenges, we introduce two novel techniques, TAQA and BSAP. TAQA is a two-stage online AQP algorithm that achieves all three properties for arbitrary queries. However, it can be slower than exact queries if we use standard row-level sampling. BSAP resolves this by enabling block-level sampling with statistical guarantees in TAQA. We simple ment TAQA and BSAP in a prototype middleware system, PilotDB, that is compatible with all DBMSs supporting efficient block-level sampling. We evaluate PilotDB on PostgreSQL, SQL Server, and DuckDB over real-world benchmarks, demonstrating up to 126X speedups when running with a 5% guaranteed error.
Keywords
Cite
@article{arxiv.2503.21087,
title = {PilotDB: Database-Agnostic Online Approximate Query Processing with A Priori Error Guarantees (Technical Report)},
author = {Yuxuan Zhu and Tengjun Jin and Stefanos Baziotis and Chengsong Zhang and Charith Mendis and Daniel Kang},
journal= {arXiv preprint arXiv:2503.21087},
year = {2025}
}
Comments
23 pages, 19 figures