Photorealistic avatars of human faces have come a long way in recent years, yet research along this area is limited by a lack of publicly available, high-quality datasets covering both, dense multi-view camera captures, and rich facial expressions of the captured subjects. In this work, we present Multiface, a new multi-view, high-resolution human face dataset collected from 13 identities at Reality Labs Research for neural face rendering. We introduce Mugsy, a large scale multi-camera apparatus to capture high-resolution synchronized videos of a facial performance. The goal of Multiface is to close the gap in accessibility to high quality data in the academic community and to enable research in VR telepresence. Along with the release of the dataset, we conduct ablation studies on the influence of different model architectures toward the model's interpolation capacity of novel viewpoint and expressions. With a conditional VAE model serving as our baseline, we found that adding spatial bias, texture warp field, and residual connections improves performance on novel view synthesis. Our code and data is available at: https://github.com/facebookresearch/multiface
@article{arxiv.2207.11243,
title = {Multiface: A Dataset for Neural Face Rendering},
author = {Cheng-hsin Wuu and Ningyuan Zheng and Scott Ardisson and Rohan Bali and Danielle Belko and Eric Brockmeyer and Lucas Evans and Timothy Godisart and Hyowon Ha and Xuhua Huang and Alexander Hypes and Taylor Koska and Steven Krenn and Stephen Lombardi and Xiaomin Luo and Kevyn McPhail and Laura Millerschoen and Michal Perdoch and Mark Pitts and Alexander Richard and Jason Saragih and Junko Saragih and Takaaki Shiratori and Tomas Simon and Matt Stewart and Autumn Trimble and Xinshuo Weng and David Whitewolf and Chenglei Wu and Shoou-I Yu and Yaser Sheikh},
journal= {arXiv preprint arXiv:2207.11243},
year = {2023}
}