Heterogeneous face re-identification, namely matching heterogeneous faces across disjoint visible light (VIS) and near-infrared (NIR) cameras, has become an important problem in video surveillance application. However, the large domain discrepancy between heterogeneous NIR-VIS faces makes the performance of face re-identification degraded dramatically. To solve this problem, a multimodal fusion ranking optimization algorithm for heterogeneous face re-identification is proposed in this paper. Firstly, we design a heterogeneous face translation network to obtain multimodal face pairs, including NIR-VIS/NIR-NIR/VIS-VIS face pairs, through mutual transformation between NIR-VIS faces. Secondly, we propose linear and non-linear fusion strategies to aggregate initial ranking lists of multimodal face pairs and acquire the optimized re-ranked list based on modal complementarity. The experimental results show that the proposed multimodal fusion ranking optimization algorithm can effectively utilize the complementarity and outperforms some relative methods on the SCface dataset.
@article{arxiv.2212.05510,
title = {Mutimodal Ranking Optimization for Heterogeneous Face Re-identification},
author = {Hui Hu and Jiawei Zhang and Zhen Han},
journal= {arXiv preprint arXiv:2212.05510},
year = {2024}
}
Comments
In the methods section, unpaired face samples should be used for training during negative sample training, rather than paired samples. Corresponding errors are also present in the subsequent experimental results