Natural Language Video Localization: A Revisit in Span-Based Question Answering Framework

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Existing Natural Language Video Localization (NLVL) methods, often vision-based, struggle with performance degradation on long videos.
• Current approaches typically frame NLVL as ranking, anchor, or regression tasks.

Purpose of the Study:

• To propose a novel approach for NLVL by adapting a span-based question answering (QA) framework.
• To develop a model, VSLNet, that effectively addresses the unique challenges of NLVL compared to traditional QA.

Main Methods:

• Treated untrimmed videos as text passages within a span-based QA framework.
• Introduced a query-guided highlighting (QGH) strategy to focus search within relevant video segments.
• Developed VSLNet-L with a multi-scale split-and-concatenation strategy to mitigate performance loss on long videos.

Main Results:

• VSLNet and VSLNet-L demonstrated superior performance over state-of-the-art methods on benchmark datasets.
• VSLNet-L effectively addressed the performance degradation issue when localizing in long videos.
• The span-based QA approach proved effective for the NLVL problem.

Conclusions:

• The span-based question answering framework offers a promising new direction for Natural Language Video Localization.
• VSLNet and its extension VSLNet-L provide effective solutions for accurate video moment localization, especially in long videos.