使用NVIDIA NeMo训练视频基础模型
计算机视觉与模式识别
2025-03-18 v1 人工智能
机器学习
摘要
视频基础模型（Video Foundation Models, VFMs）最近被用于模拟真实世界，以训练物理AI系统并开发创意视觉体验。然而，训练能够生成高质量视频的大规模、高质量VFM面临重大挑战。我们提出了一种基于NVIDIA NeMo的可扩展、开源的VFM训练流水线，提供加速的视频数据集整理、多模态数据加载，以及并行化的视频扩散模型训练和推理。我们还提供了全面的性能分析，突出了高效VFM训练和推理的最佳实践。
关键词
引用
@article{arxiv.2503.12964,
title = {Training Video Foundation Models with NVIDIA NeMo},
author = {Zeeshan Patel and Ethan He and Parth Mannan and Xiaowei Ren and Ryan Wolf and Niket Agarwal and Jacob Huffman and Zhuoyao Wang and Carl Wang and Jack Chang and Yan Bai and Tommy Huang and Linnan Wang and Sahil Jain and Shanmugam Ramasamy and Joseph Jennings and Ekaterina Sirazitdinova and Oleg Sudakov and Mingyuan Ma and Bobby Chen and Forrest Lin and Hao Wang and Vasanth Rao Naik Sabavat and Sriharsha Niverty and Rong Ou and Pallab Bhattacharya and David Page and Nima Tajbakhsh and Ashwath Aithal},
journal= {arXiv preprint arXiv:2503.12964},
year = {2025}
}