Related Experiment Video
Updated: Sep 10, 2025
04:48
Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography
Published on: November 30, 2022
2.9K
Integrating snapshot ensemble learning into masked autoencoders for efficient self-supervised pretraining in medical
1School of Bio-Health Convergence, College of Natural Sciences, Sungshin Women's University, Seoul, Republic of Korea.
Scientific Reports
|August 25, 2025
Summary
Snap-MAE integrates snapshot ensemble learning into masked autoencoder pretraining for medical imaging. This approach enhances performance and reduces computational costs by capturing diverse models in a single training phase.
Area of Science:
- Artificial Intelligence
- Medical Imaging
- Computer Vision
Background:
- Self-supervised learning (SSL) effectively utilizes unlabeled medical data for model pretraining.
- Masked autoencoders (MAE) excel at learning representations by reconstructing masked image patches.
- Training multiple MAE models for ensemble predictions is computationally intensive.
Purpose of the Study:
- To introduce Snap-MAE, a novel approach combining snapshot ensemble learning with MAE pretraining.
- To improve computational efficiency and performance in medical imaging model pretraining.
- To address the resource limitations often encountered in medical AI research.
Main Methods:
- Snap-MAE employs a cyclic cosine scheduler for learning rate adjustment during pretraining.
- Snapshot ensemble learning is integrated to capture diverse model representations within one training cycle.
- Snapshot models are fine-tuned on labeled data and ensembled for final predictions.
Main Results:
- Snap-MAE consistently outperformed vanilla MAE, ViT-S, and ResNet-34 on pediatric thoracic disease classification and cardiovascular disease diagnosis.
- The method demonstrated superior performance across all evaluated metrics.
- Snap-MAE significantly reduced the computational burden compared to traditional ensemble methods.
Conclusions:
- Snap-MAE offers a computationally efficient and effective solution for SSL-based pretraining in medical imaging.
- The model's ability to generate diverse pretrained models from a single phase makes it practical for resource-constrained environments.
- Snap-MAE represents a significant advancement for leveraging unlabeled data in medical AI.
