MaRI: Accelerating Ranking Model Inference via Structural Re-parameterization in Large Scale Recommendation System
Abstract
Ranking models, i.e., coarse-ranking and fine-ranking models, serve as core components in large-scale recommendation systems, responsible for scoring massive item candidates based on user preferences. To meet the stringent latency requirements of online serving, structural lightweighting or knowledge distillation techniques are commonly employed for ranking model acceleration. However, these approaches typically lead to a non-negligible drop in accuracy. Notably, the angle of lossless acceleration by optimizing feature fusion matrix multiplication, particularly through structural reparameterization, remains underexplored. In this paper, we propose MaRI, a novel Matrix Re-parameterized Inference framework, which serves as a complementary approach to existing techniques while accelerating ranking model inference without any accuracy loss. MaRI is motivated by the observation that user-side computation is redundant in feature fusion matrix multiplication, and we therefore adopt the philosophy of structural reparameterization to alleviate such redundancy.
Cite
@article{arxiv.2602.23105,
title = {MaRI: Accelerating Ranking Model Inference via Structural Re-parameterization in Large Scale Recommendation System},
author = {Yusheng Huang and Pengbo Xu and Shen Wang and Changxin Lao and Jiangxia Cao and Shuang Wen and Shuang Yang and Zhaojie Liu and Han Li and Kun Gai},
journal= {arXiv preprint arXiv:2602.23105},
year = {2026}
}
Comments
Work in progress