Related Experiment Video
Updated: Jun 28, 2025
04:48
Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique
Published on: July 5, 2024
394
MoViT: Memorizing Vision Transformers for Medical Image Analysis
Yiqing Shen1, Pengfei Guo1, Jingpu Wu1
1Johns Hopkins University, Baltimore, USA.
Machine Learning in Medical Imaging. MLMI (Workshop)
|April 15, 2024
Summary
Memorizing Vision Transformer (MoViT) reduces the need for large datasets in medical imaging AI. This approach uses external memory to train transformer models effectively, even with limited data, achieving competitive performance with significantly less training.
Area of Science:
- Artificial Intelligence
- Medical Image Analysis
- Computer Vision
Background:
- Transformers and CNNs offer complementary benefits in medical image analysis.
- Transformers require substantial training data, posing challenges in medical imaging due to data limitations.
Purpose of the Study:
- To propose a novel Memorizing Vision Transformer (MoViT) to reduce the reliance on large datasets for training transformer-based medical image analysis models.
- To enhance the efficiency and applicability of transformer architectures in data-scarce medical imaging scenarios.
Main Methods:
- MoViT employs an external memory to cache attention snapshots during training.
- An attention temporal moving average scheme is used to prevent overfitting by updating memories.
- Prototypical attention learning is utilized for inference speedup by distilling memory into smaller subsets.
Main Results:
- MoViT outperforms vanilla transformer models on histology and MRI datasets, particularly with limited annotated data.
- The proposed method achieves competitive performance comparable to Vision Transformer (ViT) using only 3.0% of the training data.
- Demonstrated effectiveness across varied medical image analysis tasks.
Conclusions:
- MoViT serves as a plug-in solution to significantly decrease the training data requirements for transformer architectures in medical image analysis.
- The approach facilitates the development of effective AI models for medical imaging even when data availability is constrained.
