RealCam：基于自回归框架的实时新视角视频生成
摘要
相机控制的视频到视频（V2V）生成通过单目录像实现动态视点合成，具有巨大的潜力，用于交互式电影制作和现场转播。然而，现有隐式合成方法根本依赖于非因果的全序列处理和刚性前缀式时间拼接。这种架构范式要求双向注意力，导致计算延迟不可接受、复杂度呈二次增长，并且本质上与实时流式或可变长度输入不兼容。为克服这些限制，我们引入了 RealCam，一种用于交互式、实时相机控制 V2V 生成的新型自回归框架。我们首先设计了一个基于跨帧情境学习（Cross-frame In-context Learning）范式的高保真教师模型。通过将源帧和目标帧交错拼接为同步的情境对，Our design inherently enables length-agnostic generalization and naturally facilitates causal adaptation, breaking the rigid prefix bottleneck. We then distill this teacher into a few-step causal student via Self-Forcing with Distribution Matching Distillation, enabling efficient, on-the-fly streaming synthesis. Furthermore, to mitigate severe loop inconsistency in closed-loop trajectories, we propose Loop-Closed Data Augmentation (LoopAug), a novel paradigm that synthesizes globally consistent loop sequences from existing multiview datasets. Extensive experiments demonstrate that RealCam achieves state-of-the-art visual fidelity and temporal consistency while enabling truly interactive camera control with orders-of-magnitude faster inference than existing paradigms. Our project page is at https://xyc-fly.github.io/RealCam/.
引用
@article{arxiv.2605.06051,
title = {RealCam: Real-Time Novel-View Video Generation with Interactive Camera Control},
author = {Youcan Xu and Jiaxin Shi and Zhen Wang and Wensong Song and Feifei Shao and Chen Liang and Jun Xiao and Long Chen},
journal= {arXiv preprint arXiv:2605.06051},
year = {2026}
}