Estimating heading from optic flow: Comparing deep learning network and human performance.

Area of Science:

• Neuroscience
• Computer Vision
• Computational Neuroscience

Background:

• Convolutional neural networks (CNNs) excel at visual recognition, modeling the brain's ventral visual stream.
• Human heading perception, crucial for self-motion, relies on the dorsal visual stream and optic flow.
• The efficacy of CNNs in modeling dorsal stream functions like heading perception remains largely unexplored.

Purpose of the Study:

• To evaluate the accuracy of CNNs in estimating human heading perception from optic flow.
• To investigate whether recurrent processing enhances CNN performance in heading estimation tasks.
• To compare CNN-based heading perception with human performance across various simulated self-motion scenarios.

Main Methods:

• Assessed CNN accuracy in estimating heading from optic flow under conditions like sparse flow, moving objects, and rotation.
• Simulated self-motion using minimal and realistic visual scenes.
• Compared CNN performance with established human heading perception data.

Main Results:

• Standard CNNs showed limited accuracy in replicating human heading perception.
• Incorporating recurrent processing into CNNs substantially improved heading estimation performance.
• Recurrent CNNs demonstrated near human-like accuracy in several self-motion scenarios.

Conclusions:

• Standard CNNs do not fully capture the mechanisms of human heading perception.
• Recurrent processing is critical for achieving human-like heading estimation accuracy in artificial systems.
• This research highlights the importance of recurrent dynamics in modeling dorsal stream functions related to self-motion.