We propose a self-supervised approach for training multi-frame video denoising networks. These networks predict frame t from a window of frames around t. Our self-supervised approach benefits from the video temporal consistency by penalizing a loss between the predicted frame t and a neighboring target frame, which are aligned using an optical flow. We use the proposed strategy for online internal learning, where a pre-trained network is fine-tuned to denoise a new unknown noise type from a single video. After a few frames, the proposed fine-tuning reaches and sometimes surpasses the performance of a state-of-the-art network trained with supervision. In addition, for a wide range of noise types, it can be applied blindly without knowing the noise distribution. We demonstrate this by showing results on blind denoising of different synthetic and realistic noises.
@article{arxiv.2004.06957,
title = {Self-Supervised training for blind multi-frame video denoising},
author = {Valéry Dewil and Jérémy Anger and Axel Davy and Thibaud Ehret and Pablo Arias and Gabriele Facciolo},
journal= {arXiv preprint arXiv:2004.06957},
year = {2021}
}