Seaweed-7B:面向视频生成基础模型的高性价比训练
计算机视觉与模式识别
2025-05-06 v2 人工智能
摘要
本技术报告提出了一种用于训练视频生成基础模型的高性价比策略。我们展示了一个约 70 亿参数(7B)的中等规模研究模型 Seaweed-7B,该模型从零开始训练,共使用 665,000 H100 GPU 小时。尽管训练所用的计算资源适中,Seaweed-7B 仍展现出与规模远大于它的当代视频生成模型相比极具竞争力的性能。在资源受限的条件下,设计选择尤为关键。本技术报告重点阐述了提升中等规模扩散模型性能的关键设计决策。在实验中,我们得出两点观察:(1) Seaweed-7B 取得了与在显著更多 GPU 资源上训练的更大模型相当甚至更优的性能;(2) 我们的模型具有强大的泛化能力,可通过轻量化微调或继续训练有效地适配广泛的下游应用。详见项目页面 https://seaweed.video/
引用
@article{arxiv.2504.08685,
title = {Seaweed-7B: Cost-Effective Training of Video Generation Foundation Model},
author = {Team Seawead and Ceyuan Yang and Zhijie Lin and Yang Zhao and Shanchuan Lin and Zhibei Ma and Haoyuan Guo and Hao Chen and Lu Qi and Sen Wang and Feng Cheng and Feilong Zuo and Xuejiao Zeng and Ziyan Yang and Fangyuan Kong and Meng Wei and Zhiwu Qing and Fei Xiao and Tuyen Hoang and Siyu Zhang and Peihao Zhu and Qi Zhao and Jiangqiao Yan and Liangke Gui and Sheng Bi and Jiashi Li and Yuxi Ren and Rui Wang and Huixia Li and Xuefeng Xiao and Shu Liu and Feng Ling and Heng Zhang and Houmin Wei and Huafeng Kuang and Jerry Duncan and Junda Zhang and Junru Zheng and Li Sun and Manlin Zhang and Renfei Sun and Xiaobin Zhuang and Xiaojie Li and Xin Xia and Xuyan Chi and Yanghua Peng and Yuping Wang and Yuxuan Wang and Zhongkai Zhao and Zhuo Chen and Zuquan Song and Zhenheng Yang and Jiashi Feng and Jianchao Yang and Lu Jiang},
journal= {arXiv preprint arXiv:2504.08685},
year = {2025}
}
备注
Technical report (some typos fixed)