Fast Weakly Supervised Action Segmentation Using Mutual Consistency.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Action segmentation in videos is crucial but annotation-intensive.
• Weakly supervised methods using transcripts offer a cost-effective alternative.
• Existing methods face challenges in efficiency and accuracy.

Purpose of the Study:

• To develop a novel end-to-end approach for weakly supervised action segmentation.
• To improve training and inference speed without sacrificing accuracy.
• To introduce a new mutual consistency loss for enhanced performance.

Main Methods:

• A two-branch neural network architecture is proposed.
• The network predicts two distinct yet redundant representations for action segmentation.
• A novel mutual consistency (MuCon) loss is introduced to enforce consistency between representations, alongside a transcript prediction loss.

Main Results:

• The proposed approach achieves state-of-the-art accuracy in weakly supervised action segmentation.
• Training time is reduced by 14x, and inference speed is improved by 20x compared to existing methods.
• The MuCon loss demonstrates effectiveness even in fully supervised settings.

Conclusions:

• The novel end-to-end approach with MuCon loss offers a highly efficient and accurate solution for weakly supervised action segmentation.
• This method significantly reduces computational costs associated with video analysis.
• The findings suggest broader applicability of the MuCon loss in supervised learning tasks.