A self-supervised spatio-temporal attention network for video-based 3D infant pose estimation.

Area of Science:

• Computer Vision
• Developmental Pediatrics
• Machine Learning

Background:

• Early detection of cerebral palsy (CP) in infants is critical for timely intervention.
• Existing human pose estimation methods lack sufficient infant-specific datasets and 3D pose annotations.
• Current methods primarily focus on adult pose estimation, limiting applications in infant development.

Purpose of the Study:

• To develop accurate 2D and 3D infant pose estimation models for early detection of developmental disorders.
• To address the scarcity of infant pose data by proposing self-supervised learning techniques.
• To enhance the clinical utility of pose estimation for General Movement Assessment (GMA).

Main Methods:

• Fine-tuned YOLO-infantPose for 2D infant pose estimation.
• Developed STAPose3D, a self-supervised 3D infant pose estimation model using multi-view videos.
• Employed temporal convolution, temporal attention, and graph attention for spatio-temporal feature learning.
• Implemented a two-stage approach: 2D pose estimation followed by 3D pose lifting.

Main Results:

• Fine-tuned YOLO-infantPose demonstrated superior performance on clinical and public infant datasets.
• STAPose3D effectively utilized multi-view data to improve 3D infant pose estimation accuracy.
• 3D pose estimation significantly enhanced General Movement Assessment (GMA) prediction compared to 2D methods.

Conclusions:

• The proposed YOLO-infantPose and STAPose3D models offer robust solutions for infant pose estimation.
• These models advance the potential for early detection of cerebral palsy and other developmental conditions.
• The developed methods show promise for improving clinical assessments of infant general movements.