Cross-Skeleton Interaction Graph Aggregation Network for Representation Learning of Mouse Social Behaviour.

Area of Science:

• Behavioral Neuroscience
• Computational Neuroscience
• Machine Learning

Background:

• Automated analysis of mouse social behavior is crucial in neuroscience.
• Pose information (keypoints/skeleton) is increasingly used for behavior interpretation.
• Existing methods struggle to effectively encode and decode complex social interactions from pose data due to deformable shapes and ambiguous movements.

Purpose of the Study:

• To propose a novel network, the Cross-Skeleton Interaction Graph Aggregation Network (CS-IGANet), for analyzing complex social interactions in mice using pose data.
• To effectively model multi-level interactions (intra-, inter-, and cross-skeleton) between mice.
• To enhance the representation learning of social behaviors through dynamic graph-level and node-level updates.

Main Methods:

• Developed a Cross-Skeleton Interaction Graph Aggregation Network (CS-IGANet) incorporating a Cross-Skeleton Node-level Interaction module (CS-NLI).
• Designed an Interaction-Aware Transformer (IAT) with an interaction-aware self-attention mechanism for dynamic representation learning.
• Implemented an auxiliary self-supervised learning task to measure cross-skeleton node similarity.

Main Results:

• The CS-IGANet model demonstrated superior performance in analyzing mouse social behaviors compared to state-of-the-art approaches.
• The proposed CS-NLI module effectively modeled multi-level interactions.
• The IAT, guided by self-attention, dynamically updated representations for improved accuracy.

Conclusions:

• CS-IGANet provides an effective framework for analyzing complex mouse social interactions using skeletal data.
• The novel interaction-aware modules and self-supervised task enhance the model's ability to capture subtle behavioral dynamics.
• This approach advances automated social behavior analysis in behavioral neuroscience.