A parametric texture model based on deep convolutional features closely matches texture appearance for humans.

Area of Science:

• Computer Vision
• Human Perception
• Image Synthesis

Background:

• Human visual system excels at perceiving subtle texture variations.
• Understanding image features crucial for texture perception is key for realistic image synthesis.
• Existing texture models include the Portilla and Simoncelli model and deep convolutional neural network (CNN) based models.

Purpose of the Study:

• To psychophysically compare the performance of a deep CNN (VGG-19) model against the Portilla and Simoncelli model and an extended CNN model for texture perception.
• To investigate the impact of viewing conditions (parafoveal vs. inspection) on texture discrimination.
• To identify which image features best capture human perception of natural textures.

Main Methods:

• A spatial three-alternative oddity paradigm was used to psychophysically compare model-generated textures with natural textures.
• Two viewing conditions were employed: brief parafoveal presentation and foveal inspection with eye movements.
• Models compared included a VGG-19 CNN model, the Portilla and Simoncelli model, and a CNN model with added power spectrum matching.

Main Results:

• Under parafoveal viewing, CNN and Portilla-Simoncelli models generated highly realistic textures, with observers unable to discriminate 10-11 out of 12 natural textures.
• Under foveal inspection, the CNN model significantly outperformed the Portilla-Simoncelli model in matching texture appearance.
• No model could generate indistinguishable textures for all 12 natural images under inspection conditions.

Conclusions:

• Deep CNN features, particularly VGG-19, are effective for synthesizing textures that are often perceptually indistinguishable from natural textures.
• Texture model performance is dependent on viewing conditions, with CNNs showing advantages during detailed inspection.
• Current models, including advanced CNNs, do not uniformly capture all aspects of human texture perception across diverse textures and viewing scenarios.