Large multimodal models (LMMs) have achieved impressive performance on various vision-language tasks, but their substantial computational and memory costs hinder their practical deployment. Existing compression methods often decouple low-rank decomposition and quantization, leading to compounded reconstruction errors, especially in multimodal architectures with cross-modal redundancy. To address this issue, we propose LLaVA-FA, a novel efficient LMM that performs joint low-rank plus quantization approximation in the frequency domain. By leveraging the de-correlation and conjugate symmetry properties of Fourier transform, LLaVA-FA achieves more compact and accurate weight representations. Furthermore, we introduce PolarQuant, a polar-coordinate quantization method tailored for complex matrices, and an optional diagonal calibration (ODC) scheme that eliminates the need for large-scale calibration data. Extensive experimental results demonstrate that our proposed LLaVA-FA outperforms existing efficient multimodal models across multiple benchmarks while maintaining minimal activated parameters and low computational costs, validating its effectiveness as a powerful solution for compressing LMMs.
@article{arxiv.2602.00135,
title = {LLaVA-FA: Learning Fourier Approximation for Compressing Large Multimodal Models},
author = {Pengcheng Zheng and Chaoning Zhang and Jiarong Mo and GuoHui Li and Jiaquan Zhang and Jiahao Zhang and Sihan Cao and Sheng Zheng and Caiyan Qin and Guoqing Wang and Yang Yang},
journal= {arXiv preprint arXiv:2602.00135},
year = {2026}
}