Shape-Based Measures Improve Scene Categorization

Area of Science:

• Computer Vision
• Cognitive Science
• Artificial Intelligence

Background:

• Deep neural networks (DNNs) exhibit bias towards color and texture, neglecting contour information in image analysis.
• Humans excel at object and scene recognition using contours, guided by Gestalt grouping principles.
• Computational models lack implementations of perceptual grouping rules for mid-level vision.

Purpose of the Study:

• To develop novel algorithms for detecting contour-based cues in complex scenes.
• To computationally implement Gestalt grouping rules for mid-level vision.
• To assess the impact of contour cues on scene categorization accuracy.

Main Methods:

• Developed algorithms for detecting contour-based cues in complex scenes.
• Utilized the medial axis transform (MAT) to score contours based on grouping rules.
• Evaluated scene categorization with and without emphasized perceptual grouping information.

Main Results:

• Both human observers and Convolutional Neural Network (CNN) models achieved higher accuracy when perceptual grouping information was emphasized.
• Weighting contours with the novel measures significantly boosted CNN model performance compared to unweighted contours.
• Current CNN models do not appear to extract or utilize these contour-based grouping cues despite their importance.

Conclusions:

• Contour-based perceptual grouping cues are crucial for accurate scene categorization.
• Computational implementation of Gestalt grouping rules enhances AI model performance.
• DNNs may need architectural or training modifications to better leverage contour information.