Motor-related signals support localization invariance for stable visual perception.

Area of Science:

• Neuroscience
• Computer Vision
• Computational Neuroscience

Background:

• Perceiving a stable visual world despite constant body and eye movements is a long-standing neuroscience challenge.
• Hierarchical convolutional neural networks (CNNs), inspired by mammalian visual systems, offer a framework to model this problem.

Purpose of the Study:

• To investigate perceptual stability as an optimization process within CNNs for image classification.
• To understand how movement signals influence visual processing and classification accuracy.

Main Methods:

• Reformulated the perceptual stability problem within hierarchical CNN architectures.
• Introduced movement signals multiplexed with visual inputs in convolutional layers.
• Analyzed classification invariance, learning speed, and robustness to noise.

Main Results:

• Movement signals enhanced classification invariance for shifted images, leading to faster learning and improved robustness.
• Emergence of category-specific activity manifolds in late CNN layers.
• Network units exhibited movement-associated activity modulations, consistent with experimental observations during saccades.

Conclusions:

• Proposed a computational framework unifying biological observations on perceptual stability.
• Demonstrated that optimality principles in CNNs can explain visual stability under movement.
• Highlighted the role of movement signals in achieving robust image classification.