Leveraging 2D Deep Learning ImageNet-trained models for Native 3D Medical Image Analysis

Area of Science:

• Artificial Intelligence in Medical Imaging
• Computer Vision
• Deep Learning for Healthcare

Background:

• Convolutional Neural Networks (CNNs) excel in 2D computer vision but struggle with 3D medical data due to limited dataset size and diversity.
• Transfer learning offers a solution by adapting models trained on one task to another, crucial for data-scarce medical imaging domains.

Purpose of the Study:

• To explore the efficacy of using 2D pre-trained models as a foundation for 3D medical imaging applications.
• To introduce and evaluate Axial-Coronal-Sagittal (ACS) convolutions as an alternative to native 3D convolutions within the Generally Nuanced Deep Learning Framework (GaNDLF).

Main Methods:

• Incorporated ACS convolutions into the GaNDLF framework, enabling the use of 2D pre-trained encoders for 3D network architectures.
• Experimentally evaluated the approach on 3D MRI data from brain tumor patients for segmentation and radiogenomic classification tasks.
• Compared performance against standard 3D convolutional neural networks without pre-training.

Main Results:

• Achieved a model size reduction of approximately 22%.
• Demonstrated an improvement in validation accuracy by approximately 33% for both segmentation and classification tasks.
• Showcased the advantage of using pre-trained 2D CNNs with ACS convolutions over non-pre-trained 3D CNNs.

Conclusions:

• Axial-Coronal-Sagittal (ACS) convolutions effectively leverage 2D pre-trained models for 3D medical imaging tasks, improving performance and reducing model size.
• This method democratizes the use of large-scale pre-trained models in healthcare AI, offering a promising avenue for advancing medical image analysis.
• The findings highlight the potential of ACS convolutions to enhance AI model development in data-limited 3D medical imaging scenarios.