Autoregressive video generation relies on history context for content consistency and storytelling. As video histories grow longer, efficiently encoding them remains an open problem - particularly for personal users and local workflows where compute and memory budgets are limited. We present a lightweight history encoder that maps long video histories into short-length embeddings, pretrained with a frame query objective that learns to attend to content features at arbitrary temporal positions. The pretraining stage provides the encoder with dense history coverage on large-scale video data; the subsequent finetuning stage adapts the pretrained encoder under an autoregressive video generation objective to establish content-level consistency. In this way, the lightweight embeddings achieve comparable performance to heavier alternatives. We evaluate the framework with ablative settings and discuss the architecture designs.
@article{arxiv.2512.23851,
title = {Pretraining Frame Preservation for Lightweight Autoregressive Video History Embedding},
author = {Lvmin Zhang and Shengqu Cai and Muyang Li and Chong Zeng and Beijia Lu and Anyi Rao and Song Han and Gordon Wetzstein and Maneesh Agrawala},
journal= {arXiv preprint arXiv:2512.23851},
year = {2026}
}