Facemap: a framework for modeling neural activity based on orofacial tracking

Area of Science:

• Neuroscience
• Computational Biology
• Behavioral Science

Background:

• Orofacial behaviors in mice generate substantial neural activity across the brain.
• Understanding the relationship between behavior and neural signals requires advanced computational models.

Purpose of the Study:

• To develop a computational framework for accurately tracking orofacial behaviors and predicting neural activity.
• To improve the characterization of behavior-neural signal dynamics.

Main Methods:

• Development of Facemap, a framework combining a keypoint tracker and a deep neural network encoder.
• High-accuracy, high-speed tracking of mouse orofacial behaviors.
• Deep neural network model for predicting neural activity from behavioral keypoints.

Main Results:

• Facemap demonstrated superior accuracy and speed compared to existing pose estimation tools.
• The model achieved near-optimal performance with minimal annotated frames, showing adaptability.
• Facemap doubled the explained variance in visual cortex neural activity compared to previous methods.
• Predicted neuronal activity clusters were spatially widespread, and behavioral features exhibited unidirectional sequential dynamics.

Conclusions:

• Facemap offers a powerful and adaptable tool for real-time behavioral analysis and neural activity prediction.
• The framework facilitates deeper insights into the function of brain-wide neural signals and their behavioral correlates.