This paper presents the first significant work on directly predicting 3D face landmarks on neural radiance fields (NeRFs). Our 3D coarse-to-fine Face Landmarks NeRF (FLNeRF) model efficiently samples from a given face NeRF with individual facial features for accurate landmarks detection. Expression augmentation is applied to facial features in a fine scale to simulate large emotions range including exaggerated facial expressions (e.g., cheek blowing, wide opening mouth, eye blinking) for training FLNeRF. Qualitative and quantitative comparison with related state-of-the-art 3D facial landmark estimation methods demonstrate the efficacy of FLNeRF, which contributes to downstream tasks such as high-quality face editing and swapping with direct control using our NeRF landmarks. Code and data will be available. Github link: https://github.com/ZHANG1023/FLNeRF.
@article{arxiv.2211.11202,
title = {FLNeRF: 3D Facial Landmarks Estimation in Neural Radiance Fields},
author = {Hao Zhang and Tianyuan Dai and Yu-Wing Tai and Chi-Keung Tang},
journal= {arXiv preprint arXiv:2211.11202},
year = {2023}
}
Comments
Hao Zhang and Tianyuan Dai contributed equally. Project website: https://github.com/ZHANG1023/FLNeRF