PHI: Bridging Domain Shift in Long-Term Action Quality Assessment via Progressive Hierarchical Instruction

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Long-term Action Quality Assessment (AQA) evaluates action performance in extended videos.
• Existing AQA methods struggle with domain shifts between pre-trained models and AQA tasks, especially with limited data.
• Fine-tuning large backbones on small AQA datasets is computationally prohibitive.

Purpose of the Study:

• To address task-level and feature-level domain shifts in long-term AQA.
• To propose a novel method, Progressive Hierarchical Instruction (PHI), to mitigate these domain shifts.
• To improve the quantitative performance evaluation of actions in long videos.

Main Methods:

• Progressive Hierarchical Instruction (PHI) framework.
• Gap Minimization Flow (GMF) using flow matching for progressive feature domain gap reduction.
• Temporally-enhanced attention module for capturing long-range dependencies.
• List-wise Contrastive Regularization (LCR) for coarse-to-fine feature alignment and domain shift mitigation.

Main Results:

• PHI effectively reduces feature-level domain shifts, which are critical for AQA.
• The temporally-enhanced attention module captures essential long-range dependencies.
• List-wise Contrastive Regularization aids in learning fine-grained cues.
• PHI achieves state-of-the-art results on three benchmark long-term AQA datasets.

Conclusions:

• PHI provides an effective solution for domain shift challenges in long-term AQA.
• The proposed methods significantly improve action quality assessment performance.
• PHI demonstrates superiority in handling domain shifts for quantitative action performance evaluation.