A label masked autoencoder for image-guided segmentation label completion

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• High-quality annotated data are essential for accurate image segmentation.
• Incomplete or corrupted mask annotations hinder supervised learning performance.
• Existing methods struggle with refining imperfect segmentation labels.

Purpose of the Study:

• To develop a method for refining corrupted image segmentation masks without manual annotation.
• To introduce a novel mask-reconstruction task and an associated autoencoder model.
• To improve the robustness and accuracy of image segmentation models facing noisy labels.

Main Methods:

• Proposed masked segmentation label modeling (MSLM) for refining partially occluded labels.
• Introduced the label masked autoencoder (L-MAE) to identify and reconstruct erroneous regions.
• Integrated an image patch supplement (IPS) algorithm to restore missing image information.

Main Results:

• The L-MAE achieved a 4.1% improvement in average mean intersection over union (mIoU).
• Training segmentation models on L-MAE-enhanced data resulted in a 13.5% mIoU improvement on the Pascal VOC dataset.
• L-MAE attained PA-mIoU scores of 91.0% on Pascal VOC 2012 and 86.4% on Cityscapes.

Conclusions:

• The L-MAE effectively refines corrupted segmentation labels, significantly boosting segmentation performance.
• The proposed approach outperforms state-of-the-art supervised segmentation models, especially with imperfect annotations.
• This method offers a promising direction for improving segmentation models in real-world scenarios with noisy data.