视频中无监督开放词汇目标定位
计算机视觉与模式识别
2024-06-27 v2
摘要
本文中，我们表明视频表示学习与预训练视觉-语言模型的最新进展可显著改善自监督视频目标定位。我们提出一种方法，首先通过以对象为中心的方法利用槽注意力(slot attention)在视频中定位对象，然后将文本分配给所得槽。后者通过从无监督预训练CLIP模型中读取局部语义信息的无监督方式实现。所得视频目标定位除CLIP中隐含的标注外完全无监督，且它实际上是首个在常规视频基准上取得良好结果的无监督方法。
引用
@article{arxiv.2309.09858,
title = {Unsupervised Open-Vocabulary Object Localization in Videos},
author = {Ke Fan and Zechen Bai and Tianjun Xiao and Dominik Zietlow and Max Horn and Zixu Zhao and Carl-Johann Simon-Gabriel and Mike Zheng Shou and Francesco Locatello and Bernt Schiele and Thomas Brox and Zheng Zhang and Yanwei Fu and Tong He},
journal= {arXiv preprint arXiv:2309.09858},
year = {2024}
}
备注
Accepted by ICCV 2023; Presented on CVPR 2024 Workshop CORR; Project Page:https://github.com/amazon-science/object-centric-vol