Large Language Models (LLMs) deliver strong performance but are difficult to deploy under tight memory and compute constraints. Low-bit post-training quantization (PTQ) is a promising direction; however, it typically relies on calibration data, auxiliary transformations, and GPU tools. To address these limitations, we propose MSB (Multi Scale Binary), a calibration-free and transformation-free PTQ method that generalizes binary quantization to multi-bit settings. MSB optimizes a dynamic grouping criterion that minimizes within group variance, yielding group-wise multiscale levels that can be applied consistently across granularities from per tensor to block-wise configurations with 64 elements groups per row, without calibration or intermediate transforms. We implement the optimization in a CPU based solver for the quantization step and evaluate using standard bfloat16 execution without low-bit packing. On Llama 3.2 3B, MSB achieves 8.43 perplexity on WikiText-2 under 4-bit weight only block-wise quantization, compared to 7.81 in full precision and 12.23 with GPTQ its default setup. Overall, MSB provides a new optimization perspective for low-bit PTQ while simplifying the pipeline by removing calibration and transformations.
@article{arxiv.2509.03054,
title = {Calibration and Transformation-Free Weight-Only LLMs Quantization via Dynamic Grouping},
author = {Xinzhe Zheng and Zhen-Qun Yang and Zishan Liu and Haoran Xie and S. Joe Qin and Arlene Chen and Fangzhen Lin},
journal= {arXiv preprint arXiv:2509.03054},
year = {2026}
}
Comments
34 pages, 10 figures. Version 3 corrects the bit-length error and adds new experiments and analysis; the core methodology remains unchanged. Under review