DNN accelerators, significantly advanced by model compression and specialized dataflow techniques, have marked considerable progress. However, the frequent access of high-precision partial sums (PSUMs) leads to excessive memory demands in architectures utilizing input/weight stationary dataflows. Traditional compression strategies have typically overlooked PSUM quantization, which may account for 69% of power consumption. This study introduces a novel Additive Partial Sum Quantization (APSQ) method, seamlessly integrating PSUM accumulation into the quantization framework. A grouping strategy that combines APSQ with PSUM quantization enhanced by a reconfigurable architecture is further proposed. The APSQ performs nearly lossless on NLP and CV tasks across BERT, Segformer, and EfficientViT models while compressing PSUMs to INT8. This leads to a notable reduction in energy costs by 28-87%. Extended experiments on LLaMA2-7B demonstrate the potential of APSQ for large language models. Code is available at https://github.com/Yonghao-Tan/APSQ.
@article{arxiv.2505.03748,
title = {APSQ: Additive Partial Sum Quantization with Algorithm-Hardware Co-Design},
author = {Yonghao Tan and Pingcheng Dong and Yongkun Wu and Yu Liu and Xuejiao Liu and Peng Luo and Shih-Yang Liu and Xijie Huang and Dong Zhang and Luhong Liang and Kwang-Ting Cheng},
journal= {arXiv preprint arXiv:2505.03748},
year = {2025}
}