We propose an efficient method for non-rigid surface tracking from monocular RGB videos. Given a video and a template mesh, our algorithm sequentially registers the template non-rigidly to each frame. We formulate the per-frame registration as an optimization problem that includes a novel texture term specifically tailored towards tracking objects with uniform texture but fine-scale structure, such as the regular micro-structural patterns of fabric. Our texture term exploits the orientation information in the micro-structures of the objects, e.g., the yarn patterns of fabrics. This enables us to accurately track uniformly colored materials that have these high frequency micro-structures, for which traditional photometric terms are usually less effective. The results demonstrate the effectiveness of our method on both general textured non-rigid objects and monochromatic fabrics.
@article{arxiv.2107.02407,
title = {NRST: Non-rigid Surface Tracking from Monocular Video},
author = {Marc Habermann and Weipeng Xu and Helge Rhodin and Michael Zollhoefer and Gerard Pons-Moll and Christian Theobalt},
journal= {arXiv preprint arXiv:2107.02407},
year = {2021}
}