Fine-Grained Human-Centric Tracklet Segmentation with Single Frame Supervision.

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• The Fine-grained human-Centric Tracklet Segmentation (FACTS) problem involves segmenting 12 human parts per video.
• Existing methods face challenges due to small human part sizes and limited labeled data (one frame per video).
• Labeling efforts for FACTS are extensive and time-consuming.

Purpose of the Study:

• To develop an efficient method for FACTS that reduces labeling requirements.
• To leverage temporal continuity in videos for improved segmentation.
• To introduce the Temporal Context segmentation Network (TCNet) for enhanced human part segmentation.

Main Methods:

• Proposed Temporal Context segmentation Network (TCNet) utilizes both pixel-level and frame-level temporal context.
• Optical flow is employed to propagate pixel-level segmentation results across adjacent frames.
• Frame-level classification likelihood vectors are propagated to leverage temporal consistency.

Main Results:

• TCNet effectively utilizes unlabeled frames during training by exploiting temporal context.
• The method produces smooth and accurate segmentation results during inference.
• Experimental results on four video datasets demonstrate TCNet's superiority over state-of-the-art methods.

Conclusions:

• TCNet offers a robust solution for the challenging FACTS problem.
• The approach significantly reduces the need for extensive manual labeling.
• The proposed method advances the field of human part segmentation in videos.