Non-Local Video Denoising by CNN
Abstract
Non-local patch based methods were until recently state-of-the-art for image denoising but are now outperformed by CNNs. Yet they are still the state-of-the-art for video denoising, as video redundancy is a key factor to attain high denoising performance. The problem is that CNN architectures are hardly compatible with the search for self-similarities. In this work we propose a new and efficient way to feed video self-similarities to a CNN. The non-locality is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a search region. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict the clean image. We apply the proposed architecture to image and video denoising. For the latter patches are searched for in a 3D spatio-temporal volume. The proposed architecture achieves state-of-the-art results. To the best of our knowledge, this is the first successful application of a CNN to video denoising.
Cite
@article{arxiv.1811.12758,
title = {Non-Local Video Denoising by CNN},
author = {Axel Davy and Thibaud Ehret and Jean-Michel Morel and Pablo Arias and Gabriele Facciolo},
journal= {arXiv preprint arXiv:1811.12758},
year = {2019}
}
Comments
A shorter version of this work has been accepted at ICIP 2019 (A NON-LOCAL CNN FOR VIDEO DENOISING). The results of v2 were improved compared to v1 and the code was updated accordingly. Code is available at: https://github.com/axeldavy/vnlnet