Effective and Efficient Mixed Precision Quantization of Speech Foundation Models
Abstract
This paper presents a novel mixed-precision quantization approach for speech foundation models that tightly integrates mixed-precision learning and quantized model parameter estimation into one single model compression stage. Experiments conducted on LibriSpeech dataset with fine-tuned wav2vec2.0-base and HuBERT-large models suggest the resulting mixed-precision quantized models increased the lossless compression ratio by factors up to 1.7x and 1.9x over the respective uniform-precision and two-stage mixed-precision quantized baselines that perform precision learning and model parameters quantization in separate and disjointed stages, while incurring no statistically word error rate (WER) increase over the 32-bit full-precision models. The system compression time of wav2vec2.0-base and HuBERT-large models is reduced by up to 1.9 and 1.5 times over the two-stage mixed-precision baselines, while both produce lower WERs. The best-performing 3.5-bit mixed-precision quantized HuBERT-large model produces a lossless compression ratio of 8.6x over the 32-bit full-precision system.
Cite
@article{arxiv.2501.03643,
title = {Effective and Efficient Mixed Precision Quantization of Speech Foundation Models},
author = {Haoning Xu and Zhaoqing Li and Zengrui Jin and Huimeng Wang and Youjun Chen and Guinan Li and Mengzhe Geng and Shujie Hu and Jiajun Deng and Xunying Liu},
journal= {arXiv preprint arXiv:2501.03643},
year = {2025}
}
Comments
To appear at IEEE ICASSP 2025