Nova-LSM: A Distributed, Component-based LSM-tree Key-value Store
Abstract
The cloud infrastructure motivates disaggregation of monolithic data stores into components that are assembled together based on an application's workload. This study investigates disaggregation of an LSM-tree key-value store into components that communicate using RDMA. These components separate storage from processing, enabling processing components to share storage bandwidth and space. The processing components scatter blocks of a file (SSTable) across an arbitrary number of storage components and balance load across them using power-of-d. They construct ranges dynamically at runtime to parallelize compaction and enhance performance. Each component has configuration knobs that control its scalability. The resulting component-based system, Nova-LSM, is elastic. It outperforms its monolithic counterparts, both LevelDB and RocksDB, by several orders of magnitude with workloads that exhibit a skewed pattern of access to data.
Keywords
Cite
@article{arxiv.2104.01305,
title = {Nova-LSM: A Distributed, Component-based LSM-tree Key-value Store},
author = {Haoyu Huang and Shahram Ghandeharizadeh},
journal= {arXiv preprint arXiv:2104.01305},
year = {2021}
}