Evolved Hierarchical Masking for Self-Supervised Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Existing Masked Image Modeling (MIM) methods utilize fixed mask patterns, limiting their ability to model diverse visual cues.
• A static masking approach restricts the capacity of self-supervised learning models to capture comprehensive image information.

Purpose of the Study:

• To introduce an evolved hierarchical masking method for general visual cue modeling in self-supervised learning.
• To enhance the performance of vision models across various downstream tasks by dynamically adapting mask patterns during training.

Main Methods:

• Leveraging the vision model being trained to parse visual cues into a hierarchy.
• Generating dynamic mask patterns based on the learned hierarchy, evolving from low-level to high-level features.
• Implementing an efficient training process by adjusting difficulty through evolving masks, without requiring pre-trained models or annotations.

Main Results:

• Substantial performance improvements across seven diverse downstream tasks, including image retrieval, classification, and semantic segmentation.
• Outperforming the recent Masked Autoencoder (MAE) by 1.1% in ImageNet-1K classification and 1.4% in ADE20K segmentation with identical training epochs.
• Demonstrating enhanced perception of intricate details for low-level feature recognition and bridging the gap for semantic-demanding tasks.

Conclusions:

• The evolved hierarchical masking method offers a more general and effective approach to visual cue modeling in self-supervised learning.
• The dynamic masking strategy adapts to the model's learning progress, leading to superior performance and training efficiency.
• This method aligns with large language model (LLM) research, improving both detailed perception and semantic understanding in vision tasks.