Characterization of RGB-Polarization Sensor-Based Cameras

Area of Science:

• Computer Vision
• Optical Engineering
• Machine Learning

Background:

• Trichromatic RGB color filter array and polarization layer (RGB-P) sensors offer combined color and polarization imaging.
• Accurate reconstruction of scene element color and polarization is crucial for computer vision tasks.
• Existing RGB-P sensors exhibit spectral responsivity variations and chromaticity shifts under polarized light, leading to inaccurate feature estimation.

Purpose of the Study:

• To develop a robust characterization method for RGB-P imaging devices.
• To address inaccuracies in color and polarization estimation caused by spectral sensitivities and chromaticity shifts.
• To present a neural-network-based model for enhanced color and polarization feature reconstruction.

Main Methods:

• Spectral responsivity measurements were performed on RGB-P sensors.
• A chromaticity shift model for polarized irradiance was developed.
• A neural network model was trained considering spectral sensitivity for polarized irradiance and linear combination of polarization channels for visualization.
• Models were trained using natural and synthetic reflectance sets under common lighting conditions.

Main Results:

• Spectral responsivity measurements revealed different sensitivities across color and polarization channels.
• A chromaticity shift was observed and modeled for polarized irradiance, impacting color and polarization estimation.
• The neural network model demonstrated robust performance in reconstructing color and polarization features.
• The proposed method effectively overcomes limitations of existing RGB-P sensors.

Conclusions:

• The developed characterization method and neural network model enable accurate estimation of color and polarization features from RGB-P imaging devices.
• This technique significantly improves object surface color rendering in applications like photography and machine vision.
• The findings pave the way for more advanced computer vision systems leveraging polarization information.