外部大型基础模型：如何高效服务万亿参数用于在线广告推荐
信息检索
2025-07-15 v7 人工智能
机器学习
摘要
广告推荐是在线广告系统的一项重要服务，且已被积极研究。最近的研究表明，推荐模型的规模扩大和先进设计可以带来显著的性能提升。然而，随着模型规模的扩大，此类先前研究与工业界之间的差距显著增大，因为它们往往忽略了工业规模应用中的两个基本挑战。首先，模型的训练和推理预算受到限制，超出预算可能会引发延迟并损害用户体验。其次，海量数据以流式模式到达，且数据分布动态变化，因为新用户/广告加入系统而现有用户/广告离开系统。我们提出了外部大型基础模型框架来解决这些被忽视的挑战。具体而言，我们开发了外部蒸馏和数据增强系统来控制训练/推理的计算成本，同时保持高性能。我们将教师设计为类似基础模型的形式，它可以作为垂直模型服务多个学生模型，从而摊销其构建成本。我们提出了辅助头和学生适配器，以缓解由流式数据问题导致的 FM 与 VMs 之间的数据分布差距。在内部工业规模应用和公开数据集上的综合实验证明了 ExFM 带来的显著性能提升。
引用
@article{arxiv.2502.17494,
title = {External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation},
author = {Mingfu Liang and Xi Liu and Rong Jin and Boyang Liu and Qiuling Suo and Qinghai Zhou and Song Zhou and Laming Chen and Hua Zheng and Zhiyuan Li and Shali Jiang and Jiyan Yang and Xiaozhen Xia and Fan Yang and Yasmine Badr and Ellie Wen and Shuyu Xu and Hansey Chen and Zhengyu Zhang and Jade Nie and Chunzhi Yang and Zhichen Zeng and Weilin Zhang and Xingliang Huang and Qianru Li and Shiquan Wang and Evelyn Lyu and Wenjing Lu and Rui Zhang and Wenjun Wang and Jason Rudy and Mengyue Hang and Kai Wang and Yinbin Ma and Shuaiwen Wang and Sihan Zeng and Tongyi Tang and Xiaohan Wei and Longhao Jin and Jamey Zhang and Marcus Chen and Jiayi Xu and Angie Huang and Xihuan Zeng and Chi Zhang and Zhengli Zhao and Jared Yang and Qiang Jin and Xian Chen and Amit Anand Amlesahwaram and Lexi Song and Liang Luo and Yuchen Hao and Nan Xiao and Yavuz Yetim and Luoshang Pan and Gaoxiang Liu and Yuxi Hu and Yuzhen Huang and Jackie Xu and Rich Zhu and Xin Zhang and Yiqun Liu and Hang Yin and Yuxin Chen and Buyun Zhang and Xiaoyi Liu and Xingyuan Wang and Wenguang Mao and Zhijing Li and Zhehui Zhou and Feifan Gu and Qin Huang and Chonglin Sun and Nancy Yu and Shuo Gu and Shupin Mao and Benjamin Au and Jingzheng Qin and Peggy Yao and Jae-Woo Choi and Bin Gao and Ernest Wang and Lei Zhang and Wen-Yen Chen and Ted Lee and Yujie Zha and Yi Meng and Alex Gong and Edison Gao and Jack Hsueh and Jie Zheng and Alireza Vahdatpour and Yiping Han and Yantao Yao and Toshinari Kureha and Shuo Chang and Musharaf Sultan and John Bocharov and Sagar Chordia and Xiaorui Gan and Peng Sun and Rocky Liu and Bo Long and Wenlin Chen and Santanu Kolay and Huayu Li},
journal= {arXiv preprint arXiv:2502.17494},
year = {2025}
}
备注
Accepted by the ACM Web Conference (WWW) 2025 Industrial Track as Oral Presentation