HOVER: Hyperbolic Video-Text Retrieval.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Video-text retrieval is vital for computer vision applications.
• Existing methods struggle with complex action compositions in videos.
• Action-compositional videos are common but under-researched, causing performance issues.

Purpose of the Study:

• To address the limitations of current video-text retrieval methods for complex action compositions.
• To introduce a novel approach, Hyperbolic Video-text Retrieval (HOVER), for enhanced video-text understanding.
• To leverage hyperbolic geometry for modeling hierarchical semantic relationships in video and text data.

Main Methods:

• Developed HOVER, embedding videos and texts into a low-dimensional hyperbolic space.
• Longitudinally decomposed videos into hierarchical action trees, representing primitive actions and compositions.
• Encoded semantic structures and temporal dependencies using hyperbolic distance, norm, and relative cosine similarity.

Main Results:

• HOVER significantly outperformed traditional Euclidean-based methods, achieving a 28.83% performance improvement.
• Demonstrated superior performance in scenarios with limited training labels.
• Showcased strong generalization capabilities across datasets with varying action complexity.

Conclusions:

• Hyperbolic space effectively models hierarchical video-text semantics, leading to improved retrieval.
• HOVER offers a more efficient and generalizable approach for action-compositional video understanding.
• The proposed method advances the state-of-the-art in video-text retrieval for complex real-world scenarios.