We present LLaTTE (LLM-Style Latent Transformers for Temporal Events), a scalable transformer architecture for production ads recommendation. Through systematic experiments, we demonstrate that sequence modeling in recommendation systems follows predictable power-law scaling similar to LLMs. Crucially, we find that semantic features bend the scaling curve: they are a prerequisite for scaling, enabling the model to effectively utilize the capacity of deeper and longer architectures. To realize the benefits of continued scaling under strict latency constraints, we introduce a two-stage architecture that offloads the heavy computation of large, long-context models to an asynchronous upstream user model. We demonstrate that upstream improvements transfer predictably to downstream ranking tasks. Deployed as the largest user model at Meta, this multi-stage framework drives a 4.3\% conversion uplift on Facebook Feed and Reels with minimal serving overhead, establishing a practical blueprint for harnessing scaling laws in industrial recommender systems.
@article{arxiv.2601.20083,
title = {LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation},
author = {Lee Xiong and Zhirong Chen and Rahul Mayuranath and Shangran Qiu and Arda Ozdemir and Lu Li and Yang Hu and Dave Li and Jingtao Ren and Howard Cheng and Fabian Souto Herrera and Ahmed Agiza and Baruch Epshtein and Anuj Aggarwal and Julia Ulziisaikhan and Chao Wang and Dinesh Ramasamy and Parshva Doshi and Sri Reddy and Arnold Overwijk},
journal= {arXiv preprint arXiv:2601.20083},
year = {2026}
}
Comments
Lee Xiong, Zhirong Chen, and Rahul Mayuranath contributed equally to this work