速度由简单性决定：用于快速音频视频生成基石模型的单流架构
计算机视觉与模式识别
2026-03-24 v1
摘要
我们提出了 daVinci-MagiHuman, 一个开源的人类导向音频视频生成基石模型。daVinci-MagiHuman 使用单一流 Transformer 联合生成同步的视频和音频,该 Transformer 通过自注意力机制在统一的 token 序列中处理文本、视频和音频。这种单流设计避免了多流或跨注意力架构的复杂性,同时能够使用标准的训练和推理基础设施进行优化。该模型在人类导向场景中表现尤为出色,能够生成富有表现力的面部表演、自然的语音-表情配合、逼真的身体动作以及精确的音视频同步。它支持中文(普通话和粤语)、英语、日语、韩语、德语和法语等多语言语音生成。为实现高效推理,我们将单流主干与模型蒸馏、潜在空间超分辨率和 Turbo VAE 解码器相结合,在单个 H100 GPU 上可在 2 秒内生成 5 秒 256p 视频。在自动评估中,daVinci-MagiHuman 在视觉质量和文本对齐方面领先于领先的开源模型,并实现语音可理解性的最低词错误率(14.60%)。在两两人类评估中,在 2000 项比较中,daVinci-MagiHuman 对 Ovi 1.1 以 80.0% 的胜率对 LTX 2.3 以 60.9% 的胜率。我们开源了完整的模型堆栈,包括基础模型、蒸馏模型、超分辨率模型和推理代码基地。
关键词
引用
@article{arxiv.2603.21986,
title = {Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model},
author = {SII-GAIR and Sand. ai and : and Ethan Chern and Hansi Teng and Hanwen Sun and Hao Wang and Hong Pan and Hongyu Jia and Jiadi Su and Jin Li and Junjie Yu and Lijie Liu and Lingzhi Li and Lyumanshan Ye and Min Hu and Qiangang Wang and Quanwei Qi and Steffi Chern and Tao Bu and Taoran Wang and Teren Xu and Tianning Zhang and Tiantian Mi and Weixian Xu and Wenqiang Zhang and Wentai Zhang and Xianping Yi and Xiaojie Cai and Xiaoyang Kang and Yan Ma and Yixiu Liu and Yunbo Zhang and Yunpeng Huang and Yutong Lin and Zewei Tao and Zhaoliang Liu and Zheng Zhang and Zhiyao Cen and Zhixuan Yu and Zhongshu Wang and Zhulin Hu and Zijin Zhou and Zinan Guo and Yue Cao and Pengfei Liu},
journal= {arXiv preprint arXiv:2603.21986},
year = {2026}
}