Related Experiment Video
Updated: Dec 9, 2025
06:18
Mixed Reality Technology and Three-Dimensional Printing in Teaching: Heart Anatomy as an Example
Published on: April 18, 2025
622
MMTLNet: Multi-Modality Transfer Learning Network with adversarial training for 3D whole heart segmentation.
Xiangyun Liao1, Yinling Qian1, Yilong Chen1
1Shenzhen Key Laboratory of Virtual Reality and Human Interaction Technology, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China.
Summary
This study introduces a novel network for multi-modality whole heart segmentation, improving accuracy by fusing MRI and CT images using transfer learning and attention mechanisms.
Area of Science:
- Medical Imaging
- Artificial Intelligence
- Computer Vision
Background:
- Accurate whole heart segmentation (WHS) is crucial for clinical applications like surgical planning.
- Multi-modality imaging (MRI, CT) offers complementary information for improved segmentation.
- Existing methods struggle to effectively fuse information from different imaging modalities.
Purpose of the Study:
- To develop a 3D multi-modality transfer learning network (MMTLNet) for enhanced whole heart segmentation.
- To effectively fuse information from MRI and CT images for more accurate segmentation results.
- To improve the accuracy and robustness of whole heart segmentation in clinical practice.
Main Methods:
- Proposed a multi-modality transfer learning network with adversarial training (MMTLNet).
- Implemented a generator-discriminator network for domain transfer from MRI to CT.
- Integrated spatial and channel attention mechanisms within a UNet architecture.
- Introduced a novel weighted loss function for adversarial training, balancing Dice loss and generator loss.
Main Results:
- Achieved high Dice scores for whole heart segmentation: 0.914 (CT) and 0.890 (MRI).
- Demonstrated superior performance compared to state-of-the-art methods on the MM-WHS dataset.
- Successfully fused complementary information from multi-modality images for segmentation.
Conclusions:
- MMTLNet effectively fuses multi-modality image data for accurate whole heart segmentation.
- The proposed attention mechanisms and novel loss function enhance segmentation performance.
- This approach holds significant potential for improving clinical diagnosis and treatment planning.
