RAW-Adapter: Adapting Pre-Trained Visual Model to Camera RAW Images and a Benchmark

Area of Science:

• Computer Vision
• Image Signal Processing
• Machine Learning

Background:

• sRGB images are preferred for pre-training visual models due to ease of acquisition and storage.
• Camera RAW images contain richer physical details crucial for real-world scenarios.
• Existing methods often integrate image signal processing (ISP) with network modules, missing model-level synergies.

Purpose of the Study:

• To propose RAW-Adapter, a novel framework leveraging adapter-based methodologies for processing RAW image data.
• To introduce RAW-Bench, a benchmark for evaluating RAW-based computer vision algorithms.
• To enhance the performance and generalization ability of RAW-based models.

Main Methods:

• Developed RAW-Adapter, incorporating learnable ISP modules as input-level adapters.
• Integrated model-level adapters to bridge ISP processing with downstream architectures.
• Introduced RAW-Bench with 17 types of RAW-based common corruptions.
• Proposed a RAW-based data augmentation strategy.

Main Results:

• RAW-Adapter demonstrated effectiveness and efficiency compared to state-of-the-art ISP methods.
• Systematic comparison on RAW-Bench highlighted RAW-Adapter's robust performance.
• Data augmentation improved RAW-Adapter's performance and out-of-domain generalization.

Conclusions:

• RAW-Adapter offers a general framework applicable to various computer vision systems.
• The proposed method effectively utilizes the rich information in RAW images.
• RAW-Adapter shows significant promise for advancing computer vision tasks using RAW data.