The ubiquity of variable-length integers in data storage and communication necessitates efficient decoding techniques. In this paper, we present SFVInt, a simple and fast approach to decode the prevalent Little Endian Base-128 (LEB128) varints. Our approach effectively utilizes the Bit Manipulation Instruction Set 2 (BMI2) in modern Intel and AMD processors, achieving significant performance improvement while maintaining simplicity and avoiding overengineering. SFVInt, with its generic design, effectively processes both 32-bit and 64-bit unsigned integers using a unified code template, marking a significant leap forward in varint decoding efficiency. We thoroughly evaluate SFVInt's performance across various datasets and scenarios, demonstrating that it achieves up to a 2x increase in decoding speed when compared to varint decoding methods used in established frameworks like Facebook Folly and Google Protobuf.
Cite
@article{arxiv.2403.06898,
title = {SFVInt: Simple, Fast and Generic Variable-Length Integer Decoding using Bit Manipulation Instructions},
author = {Gang Liao and Ye Liu and Yonghua Ding and Le Cai and Jianjun Chen},
journal= {arXiv preprint arXiv:2403.06898},
year = {2024}
}