AVATAR: AI Vision Analysis for Three-dimensional Action in Real-time.

Area of Science:

• Neuroscience
• Computer Science
• Ethology

Background:

• Artificial intelligence (AI) offers advanced capabilities for high-resolution behavioral analysis and automated experiments.
• Current methods for analyzing animal behavior can be labor-intensive and lack real-time processing.
• There is a need for precise, automated systems to quantify complex behaviors in model organisms.

Purpose of the Study:

• To introduce AVATAR (AI Vision Analysis for Three-dimensional Action in Real-time), an AI-driven system for reconstructing and analyzing 3D mouse motion.
• To enable automated, low-latency behavioral classification and closed-loop experimental control.
• To demonstrate the system's generalizability across different behavioral paradigms.

Main Methods:

• AVATAR utilizes synchronized multi-view videos to detect key body parts and reconstruct 3D mouse poses as action skeletons.
• The system employs Long Short-Term Memory (LSTM) networks for behavior classification based on extracted kinematic and postural features.
• AVATAR incorporates low-latency processing (100 ms) for real-time applications, including closed-loop optogenetic stimulation.

Main Results:

• AVATAR achieves near-human accuracy in 3D pose estimation and robust extraction of behavioral features.
• The system reliably classifies freely moving mouse behaviors across various experimental paradigms.
• AVATAR successfully applied to a predatory hunting paradigm, enabling automated action segmentation during complex chasing behaviors.

Conclusions:

• AVATAR provides a comprehensive solution for 3D pose estimation, dynamic quantification, and real-time behavioral classification in mice.
• The system facilitates automated, human-free experiments and real-time closed-loop manipulations.
• AVATAR enhances the scalability and precision of behavioral analysis in model animal research.