Toward High Quality Multi-Object Tracking and Segmentation Without Mask Supervision.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Weakly supervised multi-object tracking and segmentation methods often suffer from coarse pseudo mask labels and underutilization of temporal information.
• Existing approaches struggle with accurate segmentation and robust tracking due to these inherent limitations.

Purpose of the Study:

• To develop a novel framework, BoxMOTS, that addresses the limitations of current weakly supervised multi-object tracking and segmentation methods.
• To eliminate the need for pseudo mask labels by directly utilizing bounding box annotations for segmentation supervision.
• To enhance the utilization of temporal information for improved mask quality and data association in tracking.

Main Methods:

• A framework that directly uses bounding box labels to supervise the segmentation network, avoiding pseudo mask labels.
• Integration of optical flow-based pairwise consistency to ensure mask consistency across frames, enhancing segmentation quality.
• A temporally adjacent pair-based sampling strategy for instance embedding learning, optimizing data association in tracking.
• An end-to-end deep model, BoxMOTS, combining these techniques for unified tracking and segmentation.

Main Results:

• BoxMOTS achieves state-of-the-art performance, significantly outperforming existing methods on benchmark datasets.
• The model demonstrates promising results on the KITTI MOTS and BDD100K MOTS datasets, validating its effectiveness.
• The proposed approach successfully utilizes only box annotations, eliminating the requirement for mask supervision.

Conclusions:

• BoxMOTS offers a more efficient and effective approach to weakly supervised multi-object tracking and segmentation.
• The framework successfully overcomes the drawbacks of coarse pseudo mask labels and limited temporal information utilization.
• The model's ability to perform accurate tracking and segmentation using only box annotations represents a significant advancement in the field.