Local Correspondence Network for Weakly Supervised Temporal Sentence Grounding.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Weakly supervised temporal sentence grounding offers practical advantages over fully supervised methods.
• Existing approaches struggle with fine-grained video-text correspondence and lack effective supervision for learning these links.

Purpose of the Study:

• To introduce an end-to-end Local Correspondence Network (LCNet) for weakly supervised temporal sentence grounding.
• To enhance the modeling of fine-grained local correspondences between video and text.

Main Methods:

• Representing video and text features hierarchically to capture detailed correspondences.
• Employing a self-supervised cycle-consistent loss to guide the learning of video-text matching.
• Utilizing self-supervised learning for exploring fine-grained correspondences, a novel approach in this domain.

Main Results:

• The proposed LCNet demonstrates significant performance improvements.
• Experiments conducted on two benchmark datasets validate the effectiveness of the LCNet approach.
• The method successfully addresses limitations in modeling fine-grained video-text correspondences.

Conclusions:

• LCNet offers a scalable and practical solution for weakly supervised temporal sentence grounding.
• The integration of hierarchical features and self-supervised learning advances the state-of-the-art.
• This work pioneers the use of self-supervised learning for fine-grained correspondence in temporal sentence grounding.