Large language model inference is often bounded by memory footprint and bandwidth in resource-constrained deployments, making quantization fundamental to efficient serving. While post-training quantization (PTQ) maintains high fidelity at 4-bit, it deteriorates at 2-3 bits. In essence, existing methods enforce a shape-invariant quantization grid (e.g., the fixed uniform intervals of UINT2) for each group, severely restricting the feasible set for error minimization. To address this, we propose Bit-Plane Decomposition Quantization (BPDQ), which constructs a variable quantization grid via bit-planes and scalar coefficients, and iteratively refines them using second-order information while progressively compensating for quantization errors to minimize output discrepancy. In the 2-bit regime, BPDQ enables serving Qwen2.5-72B on a single RTX 3090 with 83.85\% GSM8K accuracy (vs. 90.83\% at 16-bit). Moreover, we theoretically show that the variable grid expands the feasible set, and that the quantization process consistently aligns with the optimization objective in Hessian-induced geometry. The code is available at https://github.com/KingdalfGoodman/BPDQ.
@article{arxiv.2602.04163,
title = {BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models},
author = {Junyu Chen and Jungang Li and Jing Xiong and Wenjie Wang and Qingyao Yang and He Xiao and Zhen Li and Taiqiang Wu and Mengzhao Chen and Zhen Peng and Chaofan Tao and Long Shi and Hongxia Yang and Ngai Wong},
journal= {arXiv preprint arXiv:2602.04163},
year = {2026}
}