Deep Motion Prior for Weakly-Supervised Temporal Action Localization.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Weakly-Supervised Temporal Action Localization (WSTAL) uses video-level labels for action localization in untrimmed videos.
• Current WSTAL methods often use a Multi-Instance Learning (MIL) pipeline, facing challenges with motion information and training loss.
• Existing approaches inadequately leverage motion cues and employ incompatible cross-entropy losses.

Purpose of the Study:

• To address the limitations of current WSTAL methods regarding motion information utilization and training loss.
• To propose a novel approach that incorporates motion cues more effectively for improved action localization.
• To develop a new training strategy that enhances the performance of WSTAL models.

Main Methods:

• Introduced 'motionness,' a context-dependent motion prior modeled using a motion graph based on optical flow.
• Developed a motion-guided loss function to modulate network training based on motionness scores.
• Integrated the proposed methods into a standard MIL pipeline for WSTAL.

Main Results:

• Motionness effectively models actions of interest, significantly improving action localization accuracy.
• The motion-guided loss leads to more accurate results compared to standard training losses.
• The proposed plug-and-play loss function is compatible with existing WSTAL methods.

Conclusions:

• The proposed motion-guided approach enhances WSTAL by effectively utilizing motion information and optimizing training.
• The novel 'motionness' prior and motion-guided loss achieve state-of-the-art performance on benchmark datasets.
• This work offers a significant advancement in weakly-supervised action localization, with broad applicability.