Optimal Static Fully Indexable Dictionaries

Fully indexable dictionaries (FID) store sets of integer keys while supporting rank/select queries. They serve as basic building blocks in many succinct data structures. Despite the great importance of FIDs, no known FID is succinct with efficient query time when the universe size $U$ is a large polynomial in the number of keys $n$, which is the conventional parameter regime for dictionary problems. In this paper, we design an FID that uses $\log \binom{U}{n} + \frac{n}{(\log U / t)^{\Omega(t)}}$ bits of space, and answers rank/select queries in $O(t + \log \log n)$ time in the worst case, for any parameter $1 \le t \le \log n / \log \log n$, provided $U = n^{1 + \Theta(1)}$. This time-space trade-off matches known lower bounds for FIDs [P\v{a}tra\c{s}cu & Thorup STOC 2006; P\v{a}tra\c{s}cu & Viola SODA 2010] when $t \le \log^{0.99} n$. Our techniques also lead to efficient succinct data structures for the fundamental problem of maintaining $n$ integers each of $\ell = \Theta(\log n)$ bits and supporting partial-sum queries, with a trade-off between $O(t)$ query time and $n\ell + n / (\log n / t)^{\Omega(t)}$ bits of space. Prior to this work, no known data structure for the partial-sum problem achieves constant query time with $n \ell + o(n)$ bits of space usage.