BEATSCORE: Beat-Synchronous Contrastive Alignment and Event-Centric Grading for Long-Term Sports Assessment

Area of Science:

• Computer Vision
• Machine Learning
• Signal Processing

Background:

• Long-term sports assessment in video understanding is difficult due to subtle movement variations and action-music coordination.
• Existing methods struggle with weak audio-visual relationships and temporal shifts, often smoothing out critical misalignments.

Purpose of the Study:

• To introduce BEATSCORE, a beat-guided framework for explicit action-music alignment at the beat level.
• To address challenges in long-term sports video analysis by performing event-centric sparse grading.

Main Methods:

• Converting audio and motion into beat-synchronous tokens for unified rhythmic comparison.
• Employing a beat-level contrastive objective with near-offset hard negatives to detect subtle misalignments.
• Utilizing an event proposal and grading module with multiple-instance pooling for final assessment.

Main Results:

• BEATSCORE demonstrates improved accuracy on public long-term sports benchmarks.
• The framework achieves competitive efficiency in its assessments.
• Explicit beat-level alignment proves effective for subtle action-music synchronization.

Conclusions:

• BEATSCORE offers a robust solution for challenging audio-visual tasks in long-form sports videos.
• The beat-guided approach effectively captures temporal and structural cues crucial for synchrony.
• This framework advances the field of video understanding for sports analysis.