Perception of differences in naturalistic dynamic scenes, and a V1-based model

Area of Science:

• Neuroscience
• Computer Vision
• Perception

Background:

• The human visual system's ability to perceive differences in dynamic visual scenes is complex.
• Computational models aim to replicate perceptual judgments, aiding in understanding visual processing in the brain's early stages.

Purpose of the Study:

• To evaluate a computational model's ability to predict human ratings of perceptual differences between movie clips.
• To investigate the impact of spatial filtering and temporal processing on perceptual difference modeling.

Main Methods:

• Observers rated perceptual differences for 198 movie clip pairs (naturalistic, low-pass, high-pass filtered).
• Three computational models were developed: Spatial, Temporal, and Ordered-Temporal, incorporating sustained and transient impulse functions.
• Model predictions were correlated with human perceptual ratings.

Main Results:

• The Ordered-Temporal Model, incorporating naturalistic temporal filtering, significantly improved prediction accuracy (correlation from 0.507 to 0.759).
• Transient impulse functions were crucial for modeling low-pass filtered movies, while sustained functions were more important for normal/high-pass movies.
• Model performance was enhanced by including realistic temporal impulse functions.

Conclusions:

• Computational models can effectively predict human perceptual judgments of movie clip differences.
• The interplay between spatial frequency, temporal response (sustained vs. transient), and visual processing channels (V1) is critical for scene perception.
• Findings support models where sustained responses code spatial details and transient responses code dynamic events.