English

VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System

Computer Vision and Pattern Recognition 2026-07-29 v1 Artificial Intelligence

Abstract

Text-to-video models have achieved remarkable visual quality, yet they still struggle to generate physically consistent dynamics because the temporal evolution of a scene must be inferred implicitly from a highly compressed text prompt. Existing chain-of-thought approaches introduce intermediate plans or visual states, but these representations are typically non-executable or temporally sparse, limiting their ability to instantiate and control the complete spatiotemporal process. To address this limitation, we introduce VideoCoCo, an agentic dual-engine framework in which executable Blender code serves as a process-level chain of thought. Given a text prompt, a coding agent synthesizes a Blender program that explicitly specifies the scene and its temporal evolution. The executable simulation engine runs the program to produce a deterministic spatiotemporal draft, which is subsequently transformed into a photorealistic video by a generative video engine through draft-conditioned editing. This decomposition separates process-level reasoning from high-fidelity visual realization. To adapt the video editor to simulated drafts, we construct VideoCoCo-3K, a curated dataset of draft-instruction-target triplets. VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558 on PhyGenBench and from 52.18 to 77.88 on VBench-2.0, achieving the best average score on both benchmarks. These results demonstrate that executable code provides an effective, controllable, and inspectable intermediate representation for physically consistent video generation.

Cite

@article{arxiv.2607.27380,
  title  = {VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System},
  author = {Haodong Li and Tianfei Ren and Xiaoxiao Ma and Chunmei Qing and Zhen Fang and Sipeng He and Ziyu Guo and Haoyu Wu and Juanxi Tian and Yihang Zou and Ruichuan An and Dongzhi Jiang and Boxue Yang and Ji Xie and Xu Huang and Wenhao Yan and Jialv Zou and Zhengrong Yue and Yaxin Luo and Xiaotong Li and Yuzhu Wang and Junyan Ye and Jinjing Zhao and Zehui Chen and Lin Chen and Renye Yan and Feng Zhao and Pheng-Ann Heng},
  journal= {arXiv preprint arXiv:2607.27380},
  year   = {2026}
}

Comments

15 pages, 3 figures, and 3 tables