Identifying specular highlights: Insights from deep learning.

Area of Science:

• Computer Vision
• Human Perception
• Artificial Intelligence

Background:

• Specular highlights are crucial for perceiving surface gloss.
• Distinguishing highlights from texture is challenging for both humans and algorithms.
• Existing models do not accurately emulate human highlight identification.

Purpose of the Study:

• Develop an image-computable model to replicate human highlight identification.
• Create a neural network that predicts human judgments of specular highlights.
• Investigate the internal representations learned by the network.

Main Methods:

• Rendered 179,085 images of glossy, textured surfaces.
• Trained a convolutional neural network (CNN) to isolate specular reflectance.
• Used human participants to label highlights and a genetic algorithm to refine the CNN.

Main Results:

• The CNN outperformed simple intensity thresholding in predicting human highlight perception.
• A refined CNN achieved 68% shared variance with human judgments.
• Network representations showed similarity to intrinsic, geometric, and photo-geometric image features.

Conclusions:

• A CNN can effectively model human highlight identification.
• The network learns representations that capture complex visual cues for gloss.
• The model has limitations in replicating human responses to geometric constraint violations.