Selfee, self-supervised features extraction of animal behaviors

Area of Science:

• Neuroscience
• Artificial Intelligence
• Computer Vision

Background:

• Accurate animal behavior characterization is vital for neuroscience research.
• Deep learning models are widely used for behavior analysis but lack end-to-end unsupervised feature extraction for raw video data.

Purpose of the Study:

• To introduce a self-supervised feature extraction (Selfee) convolutional neural network for end-to-end analysis of animal behavior video frames.
• To enable comprehensive and discriminative feature extraction directly from raw video data for annotation and analysis.

Main Methods:

• Developed a self-supervised convolutional neural network (Selfee) for end-to-end processing of animal behavior video frames.
• Extracted features termed 'Meta-representations' for visualization, classification, and time-series analysis.

Main Results:

• Selfee extracts animal behavior features comparable to human perception.
• Meta-representations effectively detect subtle anomalous behaviors and reveal temporal dynamics.
• Demonstrated utility in multiple downstream applications for behavior analysis.

Conclusions:

• Presented a novel self-supervised learning approach (Selfee) for extracting discriminative features from raw animal behavior videos.
• Selfee offers a powerful tool for in-depth analysis, anomaly detection, and understanding behavioral temporal dynamics in neuroscience research.