Learning to Segment Human by Watching YouTube.

Area of Science:

• Computer Vision
• Machine Learning

Background:

• Human segmentation in videos is challenging due to complex motion and appearance variations.
• Existing weakly-supervised methods often rely on limited annotations like bounding boxes.

Purpose of the Study:

• To develop a very-weakly supervised learning framework for human segmentation.
• To leverage video context (appearance and motion) for improved mask inference.
• To achieve state-of-the-art performance in human segmentation using minimal supervision.

Main Methods:

• A framework combining video-context guided mask inference and CNN learning.
• Iterative refinement of human masks and segmentation network training.
• Utilizing supervoxels, graph optimization, and imperfect human detection for mask generation.
• Training CNNs with video-context derived human masks as labels.

Main Results:

• The proposed framework significantly outperforms previous weakly-supervised methods on the PASCAL VOC 2012 benchmark.
• Achieved state-of-the-art results on human segmentation when augmented with annotated masks.
• Demonstrated the effectiveness of iterative mutual enhancement between mask inference and CNN learning.

Conclusions:

• The developed very-weakly supervised framework effectively utilizes video context for human segmentation.
• The iterative approach enhances both mask quality and segmentation accuracy.
• This method offers a promising direction for semantic segmentation with limited annotations.