In-memory Incremental Maintenance of Provenance Sketches [extended version]
Abstract
Provenance-based data skipping compactly over-approximates the provenance of a query using so-called provenance sketches and utilizes such sketches to speed-up the execution of subsequent queries by skipping irrelevant data. However, a sketch captured at some time in the past may become stale if the data has been updated subsequently. Thus, there is a need to maintain provenance sketches. In this work, we introduce In-Memory incremental Maintenance of Provenance sketches (IMP), a framework for maintaining sketches incrementally under updates. At the core of IMP is an incremental query engine for data annotated with sketches that exploits the coarse-grained nature of sketches to enable novel optimizations. We experimentally demonstrate that IMP significantly reduces the cost of sketch maintenance, thereby enabling the use of provenance sketches for a broad range of workloads that involve updates.
Keywords
Cite
@article{arxiv.2505.20683,
title = {In-memory Incremental Maintenance of Provenance Sketches [extended version]},
author = {Pengyuan Li and Boris Glavic and Dieter Gawlick and Vasudha Krishnaswamy and Zhen Hua Liu and Danica Porobic and Xing Niu},
journal= {arXiv preprint arXiv:2505.20683},
year = {2025}
}