Contrastive Masked Autoencoders are Stronger Vision Learners.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Masked image modeling (MIM) shows promise in vision tasks but has limitations in representation discriminability.
• Existing methods require further improvements for developing stronger vision learners.

Purpose of the Study:

• To introduce Contrastive Masked Autoencoders (CMAE), a novel self-supervised pre-training method.
• To learn more comprehensive and capable vision representations by combining contrastive learning (CL) and MIM.

Main Methods:

• CMAE employs an asymmetric encoder-decoder (online branch) and a momentum-updated encoder (momentum branch).
• The online encoder reconstructs masked images, while the momentum encoder enhances feature discriminability through contrastive learning with full images.
• Novel components include pixel shifting for positive view generation and a feature decoder for contrastive pair feature complementation.

Main Results:

• CMAE significantly improves representation quality and transfer performance compared to standard MIM approaches.
• Achieved state-of-the-art results on image classification, semantic segmentation, and object detection benchmarks.
• CMAE-Base reached 85.3% top-1 accuracy on ImageNet and 52.5% mIoU on ADE20k, exceeding prior bests.

Conclusions:

• CMAE effectively unifies CL and MIM, leveraging their strengths for superior vision representation learning.
• The proposed method demonstrates enhanced instance discriminability and local perceptibility.
• CMAE sets new performance benchmarks, highlighting its potential for advancing computer vision.