AVATAR: AI Vision Analysis for Three-dimensional Action in Real-time

Artificial intelligence (AI) provides new opportunities for high-resolution behavioral analysis and automated, human-free experiments. Here we present AVATAR (AI Vision Analysis for Three-dimensional Action in Real-time). This AI-driven system reconstructs 3D mouse motions by detecting key body parts from synchronized multi-view videos and converting into action skeletons. AVATAR achieves near-human accuracy in pose estimation, enables robust extraction of kinematic and postural features, and supports scalable analysis of model animal behaviors. Using these features represented by 3D action skeleton, LSTM-based model reliably classifies freely moving mouse behaviors during various experimental paradigms with low-latency processing (100 ms) enables real-time closed-loop optogenetic stimulation. As a demonstration of generalizability, we applied AVATAR framework to bottom-view predatory hunting paradigm. AVATARnet accurately detected mouse poses and extracted dynamic behavioral features of the mouse. Using AVATARnet-driven dynamic features, an XGBoost-based classifier automated action segmentation annotation during complex predatory chasing behavior. Together, AVATAR provides 3D pose estimation, dynamic quantification, classification, and closed-loop manipulation in real-time.