English

DiffuseRAW: End-to-End Generative RAW Image Processing for Low-Light Images

Image and Video Processing 2024-03-01 v1 Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning

Abstract

Imaging under extremely low-light conditions presents a significant challenge and is an ill-posed problem due to the low signal-to-noise ratio (SNR) caused by minimal photon capture. Previously, diffusion models have been used for multiple kinds of generative tasks and image-to-image tasks, however, these models work as a post-processing step. These diffusion models are trained on processed images and learn on processed images. However, such approaches are often not well-suited for extremely low-light tasks. Unlike the task of low-light image enhancement or image-to-image enhancement, we tackle the task of learning the entire image-processing pipeline, from the RAW image to a processed image. For this task, a traditional image processing pipeline often consists of multiple specialized parts that are overly reliant on the downstream tasks. Unlike these, we develop a new generative ISP that relies on fine-tuning latent diffusion models on RAW images and generating processed long-exposure images which allows for the apt use of the priors from large text-to-image generation models. We evaluate our approach on popular end-to-end low-light datasets for which we see promising results and set a new SoTA on the See-in-Dark (SID) dataset. Furthermore, with this work, we hope to pave the way for more generative and diffusion-based image processing and other problems on RAW data.

Keywords

Cite

@article{arxiv.2402.18575,
  title  = {DiffuseRAW: End-to-End Generative RAW Image Processing for Low-Light Images},
  author = {Rishit Dagli},
  journal= {arXiv preprint arXiv:2402.18575},
  year   = {2024}
}
R2 v1 2026-06-28T15:03:39.239Z