VADER: Video Alignment Differencing and Retrieval

Alexander Black; Simon Jenni; Tu Bui; Md. Mehrab Tanjim; Stefano Petrangeli; Ritwik Sinha; Viswanathan Swaminathan; John Collomosse

VADER: Video Alignment Differencing and Retrieval

Computer Vision and Pattern Recognition 2023-03-28 v2

Authors: Alexander Black , Simon Jenni , Tu Bui , Md. Mehrab Tanjim , Stefano Petrangeli , Ritwik Sinha , Viswanathan Swaminathan , John Collomosse

View on arXiv ↗ PDF ↗

Abstract

We propose VADER, a spatio-temporal matching, alignment, and change summarization method to help fight misinformation spread via manipulated videos. VADER matches and coarsely aligns partial video fragments to candidate videos using a robust visual descriptor and scalable search over adaptively chunked video content. A transformer-based alignment module then refines the temporal localization of the query fragment within the matched video. A space-time comparator module identifies regions of manipulation between aligned content, invariant to any changes due to any residual temporal misalignments or artifacts arising from non-editorial changes of the content. Robustly matching video to a trusted source enables conclusions to be drawn on video provenance, enabling informed trust decisions on content encountered.

Keywords

video retrieval video understanding video generation

Cite

@article{arxiv.2303.13193,
  title  = {VADER: Video Alignment Differencing and Retrieval},
  author = {Alexander Black and Simon Jenni and Tu Bui and Md. Mehrab Tanjim and Stefano Petrangeli and Ritwik Sinha and Viswanathan Swaminathan and John Collomosse},
  journal= {arXiv preprint arXiv:2303.13193},
  year   = {2023}
}

VADER: Video Alignment Differencing and Retrieval

Abstract

Keywords

Cite

Related papers