Local cues enable classification of image patches as surfaces, object boundaries, or illumination changes

Area of Science:

• Visual perception
• Computational neuroscience
• Image processing

Background:

• Accurate visual scene parsing requires edge detection and cause identification.
• Previous research showed machine learning models sensitive to boundary sharpness and texture for edge classification.
• Human use of these specific cues for shadow versus occlusion edge classification remained uninvestigated.

Purpose of the Study:

• To investigate whether human observers utilize boundary sharpness, texture modulation, and luminance modulation cues when classifying edges as shadows or occlusions.
• To compare human edge classification strategies with those of image-computable neural networks.
• To understand the perceptual mechanisms underlying visual scene parsing.

Main Methods:

• Synthetic image patches with parametrically manipulated boundary sharpness, texture modulation, and luminance modulation were created.
• Observers were trained on natural image patches (occlusion, shadow, uniform texture).
• Observers then classified synthetic boundary images in test experiments, and performance was modeled using logistic regression.

Main Results:

• Luminance modulation and boundary sharpness exhibited strong interactions in edge classification.
• Sharp edges with increasing luminance modulation were more likely classified as occlusions.
• Blurred edges with increasing luminance modulation were more likely classified as shadows, while texture modulation had minimal impact.

Conclusions:

• Human observers utilize boundary sharpness, luminance, and texture cues for edge classification, mirroring machine learning model sensitivities.
• These findings provide insights into the neural and perceptual mechanisms of visual scene parsing.
• The study validates the use of parametric synthetic stimuli for investigating visual perception.