Low-light image enhancement (LLIE) has achieved promising performance by employing conditional diffusion models. Despite the success of some conditional methods, previous methods may neglect the importance of a sufficient formulation of task-specific condition strategy, resulting in suboptimal visual outcomes. In this study, we propose JoReS-Diff, a novel approach that incorporates Retinex- and semantic-based priors as the additional pre-processing condition to regulate the generating capabilities of the diffusion model. We first leverage pre-trained decomposition network to generate the Retinex prior, which is updated with better quality by an adjustment network and integrated into a refinement network to implement Retinex-based conditional generation at both feature- and image-levels. Moreover, the semantic prior is extracted from the input image with an off-the-shelf semantic segmentation model and incorporated through semantic attention layers. By treating Retinex- and semantic-based priors as the condition, JoReS-Diff presents a unique perspective for establishing an diffusion model for LLIE and similar image enhancement tasks. Extensive experiments validate the rationality and superiority of our approach.
@article{arxiv.2312.12826,
title = {JoReS-Diff: Joint Retinex and Semantic Priors in Diffusion Model for Low-light Image Enhancement},
author = {Yuhui Wu and Guoqing Wang and Zhiwen Wang and Yang Yang and Tianyu Li and Malu Zhang and Chongyi Li and Heng Tao Shen},
journal= {arXiv preprint arXiv:2312.12826},
year = {2024}
}