Hierarchical Prototype Alignment for Video Temporal Grounding

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Video temporal grounding performance has improved with vision-language learning.
• Existing methods often fail to align global video features with localized textual semantics.
• This limitation causes unstable alignment in complex videos with intertwined events.

Purpose of the Study:

• To propose a hierarchical prototype alignment approach for enhanced video temporal grounding.
• To jointly model fine-grained spatial semantics and temporal event structures for accurate cross-modal alignment.
• To improve the quality of cross-modal alignment and the accuracy of temporal grounding.

Main Methods:

• Developed a hierarchical prototype alignment method using structured intermediate prototypes.
• Decomposed alignment into object-phrase and event-sentence stages for fine-grained correspondence.
• Integrated cross-modal alignment information into candidate moment aggregation to focus on relevant temporal regions.

Main Results:

• The proposed method significantly outperforms existing approaches on benchmark datasets (Charades-STA, ActivityNet Captions, TACoS).
• Demonstrated improved cross-modal alignment quality.
• Achieved higher temporal grounding accuracy, especially in complex scenarios.

Conclusions:

• Hierarchical prototype alignment effectively models cross-modal correspondence between video and text.
• The method successfully addresses limitations of global feature association in video temporal grounding.
• The approach offers a promising direction for advancing vision-language understanding in video analysis.