Climbing Technique Evaluation by Means of Skeleton Video Stream Analysis

Area of Science:

• Sports Science and Biomechanics
• Computer Vision and Human-Computer Interaction

Background:

• Growing interest in climbing necessitates non-invasive motion analysis techniques.
• Existing methods often rely on invasive technology or focus on elite athletes.
• Novice climbers frequently exhibit specific, identifiable movement errors.

Purpose of the Study:

• To develop and evaluate a camera-based system for automatic detection of novice climbing errors.
• To provide real-time, actionable feedback to climbers for technique improvement.
• To utilize 3D motion analysis for enhanced accuracy in error identification.

Main Methods:

• Utilized an iPad Pro with LiDAR for capturing 3D climbing motion data.
• Employed Apple's Vision framework for 2D skeleton extraction and LiDAR for depth information.
• Developed a finite state machine to identify climbing phases and detect phase-specific errors.
• Implemented an application to provide immediate virtual mentor feedback.

Main Results:

• The system successfully identified six common novice climbing errors with sufficient accuracy for feedback.
• Precision-recall curves indicated an acceptable range of false positives.
• Limitations were identified, primarily related to LiDAR sensor range affecting joint localization.

Conclusions:

• The developed system offers a promising non-invasive solution for analyzing novice climbing technique.
• The virtual mentor application effectively delivers immediate, constructive feedback.
• Future advancements in pose estimation and sensor technology are expected to further enhance system performance.