Human-aligned evaluation of a pixel-wise DNN color constancy model.

Area of Science:

• Computer Vision
• Computational Neuroscience
• Color Science

Background:

• Investigated color constancy in photorealistic virtual reality (VR).
• Developed a Deep Neural Network (DNN) for predicting reflectance from rendered images.

Purpose of the Study:

• Compare a DNN model's performance with human color constancy mechanisms.
• Assess model and human performance using established mechanisms: local surround, maximum flux, and spatial mean.
• Evaluate model performance using an achromatic object selection task, mirroring human experiments.

Main Methods:

• Combined VR and DNN approaches for color constancy study.
• Utilized a ResNet-based U-Net model pre-trained on rendered images.
• Applied transfer learning, fine-tuning the network's decoder on VR images.
• Assessed model performance on an achromatic object selection task across conditions.

Main Results:

• Observed strong correspondence between model and human behavior.
• Both model and humans exhibited high color constancy under baseline conditions.
• Identified similar, condition-dependent performance declines when color cues were removed.

Conclusions:

• A pixel-wise DNN trained on naturalistic image statistics can replicate human color constancy.
• Supports the theory that human color constancy arises from integrating scene-based cues.
• Suggests explicit illuminant estimation may not be necessary for human color constancy.