Practice on Long Behavior Sequence Modeling in Tencent Advertising
Abstract
Long-sequence modeling has become an indispensable frontier in recommendation systems for capturing users' long-term preferences. However, user behaviors within advertising domains are inherently sparse, posing a significant barrier to constructing long behavioral sequences using data from a single advertising domain alone. This motivates us to collect users' behaviors not only across diverse advertising scenarios, but also beyond the boundaries of the advertising domain into content domains-thereby constructing unified commercial behavior trajectories. This cross-domain or cross-scenario integration gives rise to the following challenges: (1) feature taxonomy gaps between distinct scenarios and domains, (2) inter-field interference arising from irrelevant feature field pairs, and (3) target-wise interference in temporal and semantic patterns when optimizing for different advertising targets. To address these challenges, we propose several practical approaches within the two-stage framework for long-sequence modeling. In the first (search) stage, we design a hierarchical hard search method for handling complex feature taxonomy hierarchies, alongside a decoupled embedding-based soft search to alleviate conflicts between attention mechanisms and feature representation. In the second (sequence modeling) stage, we introduce: (a) Decoupled Side Information Temporal Interest Networks (TIN) to mitigate inter-field conflicts; (b) Target-Decoupled Positional Encoding and Target-Decoupled SASRec to address target-wise interference; and (c) Stacked TIN to model high-order behavioral correlations. Deployed in production on Tencent's large-scale advertising platforms, our innovations delivered significant performance gains: an overall 4.22% GMV lift in WeChat Channels and an overall 1.96% GMV increase in WeChat Moments.
Cite
@article{arxiv.2510.21714,
title = {Practice on Long Behavior Sequence Modeling in Tencent Advertising},
author = {Xian Hu and Ming Yue and Zhixiang Feng and Junwei Pan and Junjie Zhai and Ximei Wang and Xinrui Miao and Qian Li and Xun Liu and Shangyu Zhang and Letian Wang and Hua Lu and Zijian Zeng and Chen Cai and Wei Wang and Fei Xiong and Pengfei Xiong and Jintao Zhang and Zhiyuan Wu and Chunhui Zhang and Anan Liu and Jiulong You and Chao Deng and Yuekui Yang and Shudong Huang and Dapeng Liu and Haijie Gu},
journal= {arXiv preprint arXiv:2510.21714},
year = {2025}
}