We tackle the problem of estimating correspondences from a general marker, such as a movie poster, to an image that captures such a marker. Conventionally, this problem is addressed by fitting a homography model based on sparse feature matching. However, they are only able to handle plane-like markers and the sparse features do not sufficiently utilize appearance information. In this paper, we propose a novel framework NeuralMarker, training a neural network estimating dense marker correspondences under various challenging conditions, such as marker deformation, harsh lighting, etc. Besides, we also propose a novel marker correspondence evaluation method circumstancing annotations on real marker-image pairs and create a new benchmark. We show that NeuralMarker significantly outperforms previous methods and enables new interesting applications, including Augmented Reality (AR) and video editing.
@article{arxiv.2209.08896,
title = {NeuralMarker: A Framework for Learning General Marker Correspondence},
author = {Zhaoyang Huang and Xiaokun Pan and Weihong Pan and Weikang Bian and Yan Xu and Ka Chun Cheung and Guofeng Zhang and Hongsheng Li},
journal= {arXiv preprint arXiv:2209.08896},
year = {2022}
}
Comments
Accepted by ToG (SIGGRAPH Asia 2022). Project Page: https://drinkingcoder.github.io/publication/neuralmarker/