RAW-适配器：针对相机 RAW 图像的预训练视觉模型适配
摘要
sRGB 图像目前是计算机视觉研究中预训练视觉模型的主要选择，由于其获取便捷和存储效率高。与此同时，RAW 图像在可变实际困难光照条件下具有丰富的物理信息。对于基于相机 RAW 数据的计算机视觉任务，现有大多数研究采用将图像信号处理器（ISP）与后端网络集成的方法，但常常忽视了 ISP 各阶段与后续网络之间的交互能力。drawing inspiration from ongoing adapter research in NLP and CV areas, we introduce RAW-Adapter, a novel approach aimed at adapting sRGB pre-trained models to camera RAW data. RAW-Adapter comprises input-level adapters that employ learnable ISP stages to adjust RAW inputs, as well as model-level adapters to build connections between ISP stages and subsequent high-level networks. Additionally, RAW-Adapter is a general framework that could be used in various computer vision frameworks. Abundant experiments under different lighting conditions have shown our algorithm's state-of-the-art (SOTA) performance, demonstrating its effectiveness and efficiency across a range of real-world and synthetic datasets.
引用
@article{arxiv.2408.14802,
title = {RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images},
author = {Ziteng Cui and Tatsuya Harada},
journal= {arXiv preprint arXiv:2408.14802},
year = {2024}
}
备注
ECCV 2024, code link: https://github.com/cuiziteng/ECCV_RAW_Adapter