Action Quality Assessment via Hierarchical Pose-Guided Multi-Stage Contrastive Regression

Area of Science:

• Computer Vision
• Machine Learning
• Sports Analytics

Background:

• Action Quality Assessment (AQA) faces challenges due to athletes' rapid movements and subtle visual variances.
• Existing methods struggle with fine-grained pose differences and temporal continuity in multi-duration sub-actions.

Purpose of the Study:

• To develop a novel AQA method addressing limitations in capturing subtle movements and handling variable sub-action durations.
• To enhance the accuracy and fairness of automatic athletic performance evaluation.

Main Methods:

• Proposed a hierarchically pose-guided multi-stage contrastive regression approach.
• Introduced a multi-scale dynamic visual-skeleton encoder for spatio-temporal feature extraction.
• Implemented a procedure segmentation network for sub-action separation and a multi-modal fusion module.

Main Results:

• Achieved superior performance on the FineDiving and MTL-AQA datasets.
• Demonstrated the effectiveness of skeletal features over mask or auxiliary visual features.
• Introduced a new FineDiving-Pose Dataset with improved pose labels.

Conclusions:

• The proposed method significantly improves action quality assessment by effectively capturing fine-grained pose differences and respecting temporal sub-action structures.
• The novel dataset and approach offer advancements for research in automatic athletic performance evaluation.