It is not easy when evaluating 3D mapping performance because existing metrics require ground truth data that can only be collected with special instruments. In this paper, we propose a metric, dense map posterior (DMP), for this evaluation. It can work without any ground truth data. Instead, it calculates a comparable value, reflecting a map posterior probability, from dense point cloud observations. In our experiments, the proposed DMP is benchmarked against ground truth-based metrics. Results show that DMP can provide a similar evaluation capability. The proposed metric makes evaluating different methods more flexible and opens many new possibilities, such as self-supervised methods and more available datasets.
Cite
@article{arxiv.2101.10402,
title = {A metric for evaluating 3D reconstruction and mapping performance with no ground truthing},
author = {Guoxiang Zhang and YangQuan Chen},
journal= {arXiv preprint arXiv:2101.10402},
year = {2021}
}