Cross-organ, cross-modality transfer learning: feasibility study for segmentation and classification.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Deep Learning

Background:

• Convolutional Neural Networks (CNNs) are widely used in medical image analysis.
• Traditional transfer learning (TL) adapts pre-trained models to new tasks, but performance can be limited by dataset differences.
• Cross-organ, cross-modality transfer learning (XTL) explores leveraging data from diverse sources to enhance model generalizability.

Purpose of the Study:

• To evaluate the effectiveness of XTL using mammograms for improving CNN performance in medical image analysis.
• To compare XTL against traditional TL for multiple sclerosis segmentation and prostate cancer malignancy classification.

Main Methods:

• Two CNN architectures were fine-tuned using XTL, incorporating mammograms as an intermediate dataset.
• The XTL networks were applied to brain MRI segmentation (multiple sclerosis) and prostate MRI classification (tumor malignancy).
• Performance was evaluated using Dice coefficient for segmentation and Area Under the Curve (AUC) for classification, comparing XTL against traditional TL.

Main Results:

• XTL networks significantly outperformed TL networks in multiple sclerosis segmentation (Dice coefficient 0.72 vs. 0.70-0.71, p < 0.0001).
• XTL networks also showed superior performance in prostate cancer classification (AUCs 0.77-0.80 vs. 0.73-0.75, p < 0.03).
• These improvements were observed despite significant differences in image modality (X-ray vs. MRI) and dimensionality (2D vs. 3D).

Conclusions:

• XTL, utilizing mammograms, enhances CNN performance for both segmentation and classification tasks in medical imaging.
• This approach demonstrates improved transferability across different organs, modalities, and tasks compared to traditional TL.
• XTL offers a promising strategy for developing more robust and generalizable deep learning models in healthcare.