Video-to-video synthesis is a challenging problem aiming at learning a translation function between a sequence of semantic maps and a photo-realistic video depicting the characteristics of a driving video. We propose a head-to-head system of our own implementation capable of fully transferring the human head 3D pose, facial expressions and eye gaze from a source to a target actor, while preserving the identity of the target actor. Our system produces high-fidelity, temporally-smooth and photo-realistic synthetic videos faithfully transferring the human time-varying head attributes from the source to the target actor. Our proposed implementation: 1) works in real time (∼20 fps), 2) runs on a commodity laptop with a webcam as the only input, 3) is interactive, allowing the participant to drive a target person, e.g. a celebrity, politician, etc, instantly by varying their expressions, head pose, and eye gaze, and visualising the synthesised video concurrently.
@article{arxiv.2006.10500,
title = {ReenactNet: Real-time Full Head Reenactment},
author = {Mohammad Rami Koujan and Michail Christos Doukas and Anastasios Roussos and Stefanos Zafeiriou},
journal= {arXiv preprint arXiv:2006.10500},
year = {2020}
}
Comments
to be published in 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)