Reinventing 2D Convolutions for 3D Images.

Area of Science:

• Medical Imaging
• Computer Vision
• Machine Learning

Background:

• 2D and 3D approaches for 3D medical image representation learning have inherent limitations.
• 2D methods struggle with 3D context, while 3D methods lack sufficient pretraining data.
• Hybrid methods inherit disadvantages from both 2D and 3D components.

Purpose of the Study:

• To bridge the gap between 2D and 3D convolutions for medical image analysis.
• To introduce a novel method for native 3D representation learning using pretrained 2D weights.
• To enable the conversion of any 2D Convolutional Neural Network (CNN) into a 3D ACS CNN.

Main Methods:

• Proposed Axial-Coronal-Sagittal (ACS) convolutions, which split 2D kernels to process 3D data across three views.
• Demonstrated theoretical compatibility of ACS convolutions with existing 2D CNN architectures (e.g., ResNet, DenseNet).
• Utilized pretrained 2D weights for 3D representation learning within the ACS framework.

Main Results:

• Pretrained ACS CNNs consistently outperformed 2D/3D CNN counterparts in extensive experiments.
• ACS convolutions offer a plug-and-play alternative to standard 3D convolutions, even without pretraining.
• Achieved smaller model sizes and reduced computational requirements compared to standard 3D convolutions.

Conclusions:

• ACS convolutions provide a superior method for 3D medical image representation learning.
• The approach effectively leverages large-scale 2D pretraining for enhanced 3D analysis.
• ACS convolutions offer an efficient and versatile solution for medical imaging tasks.