English

Timestep-Aware SVDQuant-GPTQ for W4A4 Quantization of Wan2.2-I2V

Computer Vision and Pattern Recognition 2026-05-27 v1 Artificial Intelligence

Abstract

W4A4 quantization of large video diffusion Transformers offers substantial memory savings but is hindered by two main challenges: sparse large-magnitude activation outliers, and strongly timestep-dependent activation distributions across the multi-step denoising trajectory. These difficulties are compounded by Wan2.2-I2V's two-expert Mixture-of-Experts DiT design, whose high-noise and low-noise experts exhibit distinct quantization sensitivities that a single global calibration policy cannot capture. We propose a post-training quantization framework combining SVDQuant-based low-rank outlier compensation, GPTQ-based reconstruction-aware residual weight quantization, and timestep-bin-wise per-layer activation clipping-ratio search conducted independently for each expert. On the OpenS2V-Eval benchmark, our method reduces peak GPU memory by 59.3\% relative to the BF16 baseline while incurring only a 0.9\% drop in VBench average score and a 2.3\% drop in Imaging Quality, demonstrating that expert- and timestep-aware calibration is essential for high-fidelity W4A4 inference on MoE video DiTs.

Cite

@article{arxiv.2605.27003,
  title  = {Timestep-Aware SVDQuant-GPTQ for W4A4 Quantization of Wan2.2-I2V},
  author = {Junhao Wu and Dezhong Yao and Hai Jin},
  journal= {arXiv preprint arXiv:2605.27003},
  year   = {2026}
}
R2 v1 2026-07-22T07:34:37.050Z