Image-level supervision and self-training for transformer-based cross-modality tumor segmentation

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Deep neural networks struggle with cross-modality generalization in medical image segmentation due to limited annotated data.
• Deploying segmentation models at scale is challenging when data is scarce in both source and target imaging modalities.

Purpose of the Study:

• To propose MoDATTS, a semi-supervised training strategy for accurate cross-modality 3D tumor segmentation on unpaired bi-modal datasets.
• To improve model generalization to unannotated target modalities by leveraging image-to-image translation and self-training.

Main Methods:

• Utilized an image-to-image translation strategy (TransUNet) to generate synthetic annotated data for the target modality.
• Employed vision transformer architectures (Medformer) for segmentation and incorporated iterative self-training to bridge domain gaps.
• Integrated a semi-supervised objective using image-level labels to disentangle tumors from background, especially useful with scarce pixel-level annotations.

Main Results:

• Achieved superior performance in the CrossMoDA 2022 vestibular schwannoma segmentation challenge, with a top Dice score of 0.87±0.04.
• Demonstrated consistent Dice score improvements on a cross-modality brain glioma segmentation task (BraTS 2020 dataset).
• Reached 95% of target supervised model performance with no target annotations, and 99-100% with 20-50% target data annotation.

Conclusions:

• MoDATTS effectively addresses cross-modality generalization challenges in medical image segmentation with limited annotated data.
• The strategy significantly reduces the need for extensive pixel-level annotations in the target modality.
• MoDATTS shows strong potential for practical deployment of automated medical image segmentation models across diverse imaging datasets.