Fine-Grained Spatio-Temporal Parsing Network for Action Quality Assessment

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Action Quality Assessment (AQA) is crucial for video analysis, particularly in sports, but faces challenges due to subtle action discrepancies and holistic video representations.
• Current methods struggle to capture fine-grained intra-class variations, limiting the accuracy of AQA.

Purpose of the Study:

• To propose a novel Fine-grained Spatio-temporal Parsing Network (FSPN) for more reliable and accurate Action Quality Assessment.
• To address the limitations of holistic video representations in capturing subtle action differences.

Main Methods:

• The proposed FSPN integrates an intra-sequence action parsing module for fine-grained sub-action mining and a spatiotemporal multiscale transformer module for learning motion-oriented features and long-range dependencies.
• A group contrastive loss was designed to train the model for discriminative sub-action feature learning without explicit supervision.

Main Results:

• The FSPN demonstrated superior performance in Action Quality Assessment across FineDiving, AQA-7, and MTL-AQA datasets.
• The approach significantly outperformed state-of-the-art methods, highlighting its effectiveness and feasibility.

Conclusions:

• The Fine-grained Spatio-temporal Parsing Network (FSPN) offers a robust solution for Action Quality Assessment by effectively learning fine-grained spatiotemporal sub-action representations.
• The study validates the FSPN's ability to capture subtle action variations, leading to significant improvements in AQA accuracy.