Video Moment Retrieval With Noisy Labels.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Video Moment Retrieval (VMR) models often struggle with noisy annotations from human labeling.
• Existing VMR algorithms require clean data for effective training, which is impractical due to annotation noise.

Purpose of the Study:

• To develop a simple, effective, and end-to-end video moment retrieval framework.
• To create a model robust to noisy labels during training.
• To improve the performance of video moment retrieval systems.

Main Methods:

• Extracted multimodal features using syntactic graph convolutional networks and multihead attention, fused via cross gates and bilinear approach.
• Constructed feature pyramid networks to encode scene relationships and high semantics.
• Devised multilevel losses (frame-level with Gaussian smoothing and instance-level with model self-teaching) to mitigate noisy annotation effects.

Main Results:

• The proposed framework significantly outperforms state-of-the-art approaches on ActivityCaption and TACoS datasets.
• Demonstrated effectiveness and noise tolerance across various manual annotation noise levels.
• Achieved robust performance despite inherent noise in training data.

Conclusions:

• The developed bottom-up schema framework offers a robust solution for video moment retrieval with noisy labels.
• The multilevel loss strategy effectively handles frame-level and instance-level noise.
• The model presents a significant advancement in practical VMR applications.