CurriMAE: curriculum learning based masked autoencoders for multi-labeled pediatric thoracic disease classification

Area of Science:

• Artificial Intelligence
• Machine Learning
• Medical Imaging

Background:

• Masked autoencoders (MAE) are effective for self-supervised learning but require extensive hyperparameter tuning, particularly for the masking ratio.
• High computational costs associated with optimizing masking ratios limit MAE's practical application in medical imaging.

Purpose of the Study:

• To introduce CurriMAE, a curriculum-based training strategy for MAE that optimizes computational efficiency and model performance.
• To reduce the computational overhead of MAE pretraining by progressively increasing the masking ratio.

Main Methods:

• CurriMAE employs a curriculum learning approach, gradually increasing the masking ratio from 60% to 90% over 800 epochs.
• A cyclic cosine learning rate scheduler, resetting every 200 epochs, ensures stable convergence across four distinct training stages.
• Snapshot models are saved at the end of each stage for subsequent fine-tuning.

Main Results:

• CurriMAE demonstrated superior performance in multi-labeled pediatric thoracic disease classification compared to ResNet, ViT-S, and standard MAE.
• The proposed method achieved better classification accuracy on the PediCXR dataset after pretraining on CheXpert and ChestX-ray14.
• CurriMAE significantly reduced computational costs during the pretraining phase.

Conclusions:

• CurriMAE presents an effective and scalable self-supervised learning framework for medical imaging analysis.
• The curriculum-based progressive masking strategy offers a computationally efficient alternative to traditional MAE training.
• This approach holds promise for advancing AI applications in diagnosing pediatric thoracic diseases.