Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report
Abstract
As the quality of mobile cameras starts to play a crucial role in modern smartphones, more and more attention is now being paid to ISP algorithms used to improve various perceptual aspects of mobile photos. In this Mobile AI challenge, the target was to develop an end-to-end deep learning-based image signal processing (ISP) pipeline that can replace classical hand-crafted ISPs and achieve nearly real-time performance on smartphone NPUs. For this, the participants were provided with a novel learned ISP dataset consisting of RAW-RGB image pairs captured with the Sony IMX586 Quad Bayer mobile sensor and a professional 102-megapixel medium format camera. The runtime of all models was evaluated on the MediaTek Dimensity 1000+ platform with a dedicated AI processing unit capable of accelerating both floating-point and quantized neural networks. The proposed solutions are fully compatible with the above NPU and are capable of processing Full HD photos under 60-100 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.
Cite
@article{arxiv.2105.07809,
title = {Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report},
author = {Andrey Ignatov and Cheng-Ming Chiang and Hsien-Kai Kuo and Anastasia Sycheva and Radu Timofte and Min-Hung Chen and Man-Yu Lee and Yu-Syuan Xu and Yu Tseng and Shusong Xu and Jin Guo and Chao-Hung Chen and Ming-Chun Hsyu and Wen-Chia Tsai and Chao-Wei Chen and Grigory Malivenko and Minsu Kwon and Myungje Lee and Jaeyoon Yoo and Changbeom Kang and Shinjo Wang and Zheng Shaolong and Hao Dejun and Xie Fen and Feng Zhuang and Yipeng Ma and Jingyang Peng and Tao Wang and Fenglong Song and Chih-Chung Hsu and Kwan-Lin Chen and Mei-Hsuang Wu and Vishal Chudasama and Kalpesh Prajapati and Heena Patel and Anjali Sarvaiya and Kishor Upla and Kiran Raja and Raghavendra Ramachandra and Christoph Busch and Etienne de Stoutz},
journal= {arXiv preprint arXiv:2105.07809},
year = {2021}
}
Comments
Mobile AI 2021 Workshop and Challenges: https://ai-benchmark.com/workshops/mai/2021/