Capsule networks as recurrent models of grouping and segmentation

Area of Science:

• Computational neuroscience
• Computer vision
• Psychophysics

Background:

• Visual processing is traditionally modeled as sequential feedforward computations.
• Feedforward Convolutional Neural Networks (ffCNNs) demonstrate the efficacy of these models.
• Previous research indicated ffCNNs fail to explain human global shape processing under visual crowding.

Purpose of the Study:

• To investigate the efficacy of Capsule Neural Networks (CapsNets) in explaining human global shape processing.
• To determine the role of recurrent grouping and segmentation in visual perception.
• To compare CapsNets with ffCNNs and standard recurrent CNNs on visual crowding tasks.

Main Methods:

• Utilizing visual crowding as a controlled experimental challenge.
• Developing and testing Capsule Neural Networks (CapsNets) with recurrent capabilities.
• Comparing CapsNet performance against feedforward CNNs and standard recurrent CNNs.
• Conducting psychophysical experiments to gather human behavioral data.

Main Results:

• CapsNets successfully explain human global shape processing under visual crowding.
• ffCNNs and standard recurrent CNNs failed to account for these human visual capabilities.
• CapsNets' recurrent grouping and segmentation mechanisms were identified as crucial for success.
• Psychophysical data confirmed recurrent grouping and segmentation in human visual processing, which CapsNets accurately reproduced.

Conclusions:

• Recurrent grouping and segmentation are essential for understanding human visual processing.
• CapsNets, with their recurrent capabilities, offer a more effective model for global shape computation than traditional ffCNNs.
• The study provides converging computational and psychophysical evidence for the necessity of recurrence in visual system models.