Learning to segment self-generated from externally caused optic flow through sensorimotor mismatch circuits

Area of Science:

• Computational Neuroscience
• Systems Neuroscience
• Visual Processing

Background:

• Efficient sensory detection requires filtering irrelevant information, such as distinguishing self-motion optic flow from object-related visual input.
• Predictive coding models with sensorimotor mismatch detection offer a framework for understanding this process, with evidence in early visual areas.
• Integration of these mismatch signals into cortical networks for segmentation and categorization remains an open question.

Purpose of the Study:

• To develop a biologically plausible computational model that extends predictive coding to differentiate self-generated from externally caused optic flow.
• To investigate how sensorimotor mismatch signals are integrated within neural networks for visual input segmentation and object categorization.
• To elucidate the role of feedback connections in maintaining generative models for optic flow processing.

Main Methods:

• A three-neuron microcircuit was developed to model experience-dependent sensorimotor mismatch responses, validated against mouse calcium imaging data.
• This microcircuit was integrated into a larger neural network featuring two generative streams: one for self-motion prediction and another for external optic flow modeling.
• The model incorporated bidirectional connections between a motion-selective higher visual area (mHVA) and V1, emphasizing feedback connections for generative model maintenance.

Main Results:

• The proposed three-neuron circuit successfully replicated sensorimotor mismatch responses observed in experimental data.
• The two-stream neural network demonstrated the ability to distinguish self-generated from externally caused optic flow.
• The model showed that the mHVA learns to segment moving objects and facilitates categorization, with its architecture mapping to primate visual cortex.

Conclusions:

• The model provides a biologically plausible mechanism for separating self-motion from external visual stimuli within the visual system.
• It extends Hebbian predictive coding principles to sensorimotor contexts, enabling agents to learn and predict the sensory consequences of their own movements.
• The findings highlight the critical role of feedback connections in higher visual areas for maintaining generative models essential for visual perception and categorization.