Given unstructured videos of deformable objects, we automatically recover spatiotemporal correspondences to map one object to another (such as animals in the wild). While traditional methods based on appearance fail in such challenging conditions, we exploit consistency in object motion between instances. Our approach discovers pairs of short video intervals where the object moves in a consistent manner and uses these candidates as seeds for spatial alignment. We model the spatial correspondence between the point trajectories on the object in one interval to those in the other using a time-varying Thin Plate Spline deformation model. On a large dataset of tiger and horse videos, our method automatically aligns thousands of pairs of frames to a high accuracy, and outperforms the popular SIFT Flow algorithm.
@article{arxiv.1412.0477,
title = {Recovering Spatiotemporal Correspondence between Deformable Objects by Exploiting Consistent Foreground Motion in Video},
author = {Luca Del Pero and Susanna Ricco and Rahul Sukthankar and Vittorio Ferrari},
journal= {arXiv preprint arXiv:1412.0477},
year = {2016}
}
Comments
9 pages, 14 figures. This article is obsolete. Its contents are now covered in arXiv:1511.09319, where we discuss a comprehensive system for behavior discovery and spatial alignment of articulated object classes from unstructured video (available at https://arxiv.org/abs/1511.09319)