Recent portrait relighting methods have achieved realistic results of portrait lighting effects given a desired lighting representation such as an environment map. However, these methods are not intuitive for user interaction and lack precise lighting control. We introduce LightPainter, a scribble-based relighting system that allows users to interactively manipulate portrait lighting effect with ease. This is achieved by two conditional neural networks, a delighting module that recovers geometry and albedo optionally conditioned on skin tone, and a scribble-based module for relighting. To train the relighting module, we propose a novel scribble simulation procedure to mimic real user scribbles, which allows our pipeline to be trained without any human annotations. We demonstrate high-quality and flexible portrait lighting editing capability with both quantitative and qualitative experiments. User study comparisons with commercial lighting editing tools also demonstrate consistent user preference for our method.
Cite
@article{arxiv.2303.12950,
title = {LightPainter: Interactive Portrait Relighting with Freehand Scribble},
author = {Yiqun Mei and He Zhang and Xuaner Zhang and Jianming Zhang and Zhixin Shu and Yilin Wang and Zijun Wei and Shi Yan and HyunJoon Jung and Vishal M. Patel},
journal= {arXiv preprint arXiv:2303.12950},
year = {2023}
}