ROBUSfT: Robust Real-Time Shape-from-Template, a C++ Library
Abstract
Tracking the 3D shape of a deforming object using only monocular 2D vision is a challenging problem. This is because one should (i) infer the 3D shape from a 2D image, which is a severely underconstrained problem, and (ii) implement the whole solution pipeline in real-time. The pipeline typically requires feature detection and matching, mismatch filtering, 3D shape inference and feature tracking algorithms. We propose ROBUSfT, a conventional pipeline based on a template containing the object's rest shape, texturemap and deformation law. ROBUSfT is ready-to-use, wide-baseline, capable of handling large deformations, fast up to 30 fps, free of training, and robust against partial occlusions and discontinuity in video frames. It outperforms the state-of-the-art methods in challenging datasets. ROBUSfT is implemented as a publicly available C++ library and we provide a tutorial on how to use it in https://github.com/mrshetab/ROBUSfT
Cite
@article{arxiv.2301.04037,
title = {ROBUSfT: Robust Real-Time Shape-from-Template, a C++ Library},
author = {Mohammadreza Shetab-Bushehri and Miguel Aranda and Youcef Mezouar and Adrien Bartoli and Erol Ozgur},
journal= {arXiv preprint arXiv:2301.04037},
year = {2023}
}
Comments
This is the arXiv version of an article published in Image and Vision Computing. Please cite the accepted version: M. Shetab-Bushehri, M. Aranda, E. Ozgur, Y. Mezouar and Adrien Bartoli "ROBUSfT: Robust Real-Time Shape-from-Template, a C++ Library," in Image and Vision Computing, doi: 10.1016/j.imavis.2023.104867