Exploring biological motion perception in two-stream convolutional neural networks.

Area of Science:

• Neuroscience
• Computer Vision
• Artificial Intelligence

Background:

• Visual recognition of biological motion involves complex form and motion processing.
• Neural pathways, including dorsal and ventral streams, support this recognition.
• This biological architecture inspired the development of two-stream convolutional neural network (CNN) models.

Purpose of the Study:

• To evaluate the performance of a two-stream CNN model in recognizing biological motion.
• To compare the CNN model's capabilities with established findings in human biological motion perception.
• To identify limitations and areas for improvement in CNN-based action recognition.

Main Methods:

• A two-stream CNN model was developed, comprising spatial and temporal CNNs with a fusion network.
• The model was trained and tested using various action videos, including raw RGB and point-light formats.
• Simulations compared CNN performance against human behavioral data on biological motion perception tasks.

Main Results:

• CNNs trained on raw RGB videos performed poorly on point-light actions, necessitating transfer training.
• The models demonstrated viewpoint-dependent recognition with limited generalization.
• CNNs replicated the inversion effect with global body configuration but not with local motion signals alone.
• Partial success was observed in accounting for human perception in fine discrimination tasks with noisy inputs.

Conclusions:

• CNNs offer a computational model for aspects of biological motion perception but require tailored training data.
• Current CNN models exhibit limitations in viewpoint generalization, adaptive integration of form and motion, and incorporating specialized perceptual mechanisms.
• Further research is needed to enhance CNNs for more comprehensive and human-like biological motion recognition.