This technical report introduces Pegasus-1, a multimodal language model specialized in video content understanding and interaction through natural language. Pegasus-1 is designed to address the unique challenges posed by video data, such as interpreting spatiotemporal information, to offer nuanced video content comprehension across various lengths. This technical report overviews Pegasus-1's architecture, training strategies, and its performance in benchmarks on video conversation, zero-shot video question answering, and video summarization. We also explore qualitative characteristics of Pegasus-1 , demonstrating its capabilities as well as its limitations, in order to provide readers a balanced view of its current state and its future direction.
Cite
@article{arxiv.2404.14687,
title = {Pegasus-v1 Technical Report},
author = {Raehyuk Jung and Hyojun Go and Jaehyuk Yi and Jiho Jang and Daniel Kim and Jay Suh and Aiden Lee and Cooper Han and Jae Lee and Jeff Kim and Jin-Young Kim and Junwan Kim and Kyle Park and Lucas Lee and Mars Ha and Minjoon Seo and Abraham Jo and Ed Park and Hassan Kianinejad and SJ Kim and Tony Moon and Wade Jeong and Andrei Popescu and Esther Kim and EK Yoon and Genie Heo and Henry Choi and Jenna Kang and Kevin Han and Noah Seo and Sunny Nguyen and Ryan Won and Yeonhoo Park and Anthony Giuliani and Dave Chung and Hans Yoon and James Le and Jenny Ahn and June Lee and Maninder Saini and Meredith Sanders and Soyoung Lee and Sue Kim and Travis Couture},
journal= {arXiv preprint arXiv:2404.14687},
year = {2024}
}