Raw Spectral Filter Array Imaging for Scene Recognition

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Scene recognition identifies image environments.
• Spectral filter array cameras capture multispectral images quickly.
• Current methods demosaic raw images, adding latency and artifacts.

Purpose of the Study:

• To explore scene recognition directly on raw spectral filter array images.
• To overcome limitations imposed by demosaicing artifacts and latency.
• To develop a novel approach for multispectral image scene classification.

Main Methods:

• Utilized convolutional neural networks (CNNs) for classification.
• Developed a new raw image dataset for scene recognition.
• Employed a pretrained Places-CNN model adapted for nine spectral channels.
• Implemented a label mapping scheme for the new dataset.

Main Results:

• Scene recognition performed directly on raw images shows promise.
• Higher-resolution raw images yielded better classification performance.
• The CNN model effectively utilized all nine spectral channels.
• Pre-processing steps were evaluated for their impact on results.

Conclusions:

• Directly processing raw spectral filter array images is a viable approach for scene recognition.
• Bypassing demosaicing reduces latency and avoids associated artifacts.
• The proposed method offers a more efficient and potentially accurate alternative for scene recognition tasks.