Multispectral Snapshot Image Registration Using Learned Cross Spectral Disparity Estimation and a Deep Guided...

Area of Science:

• Computer Vision
• Image Processing
• Computational Imaging

Background:

• Multispectral imaging captures data across multiple spectral bands, crucial for applications in agriculture, recycling, and healthcare.
• Snapshot multispectral imaging using camera arrays requires precise spatial registration due to differing camera positions.

Purpose of the Study:

• To develop an advanced multispectral snapshot image registration method.
• To enhance accuracy and efficiency in aligning images from different spectral bands captured simultaneously.

Main Methods:

• A cross-spectral disparity estimation network trained with pseudo-spectral data augmentation.
• Layer-wise warping of disparity maps for accurate occlusion detection.
• Deep neural network-based reconstruction of occluded regions using information from other spectral bands.

Main Results:

• Achieved over 3 dB improvement in Peak Signal-to-Noise Ratio (PSNR) compared to state-of-the-art methods.
• Reduced runtime by over 3x on CPU and 113x on GPU.
• Demonstrated superior performance in both individual components and the overall registration process.

Conclusions:

• The proposed multispectral snapshot image registration method offers significant advancements in accuracy and speed.
• The novel deep learning components effectively address challenges like disparity estimation and occlusion handling.
• This work provides a highly efficient and accurate solution for multispectral imaging registration.