Towards cross-modal organ translation and segmentation: A cycle- and shape-consistent generative adversarial network

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Synthesized medical images are crucial for applications like cross-modality image registration and classifier training.
• Existing cross-modality synthesis methods often struggle with anatomical consistency and require paired data.

Purpose of the Study:

• To develop a generic cross-modality synthesis approach for realistic 2D/3D medical images without paired data.
• To ensure consistent anatomical structures during synthesis, mitigating geometric distortion.
• To improve volume segmentation performance using synthetic data, particularly for modalities with limited training samples.

Main Methods:

• An end-to-end 2D/3D convolutional neural network (CNN) integrating generators and segmentors.
• Generators utilize adversarial, cycle-consistency, and shape-consistency losses for realistic synthesis.
• Segmentors are enhanced online by synthetic data, with generators and segmentors mutually benefiting in an alternating training fashion.

Main Results:

• The proposed method successfully synthesizes realistic 2D/3D medical images across diverse modalities (CT, MRI, X-ray).
• Consistent anatomical structures are maintained, reducing geometric distortion.
• Significant improvements in volume segmentation accuracy were observed when using the synthetic data, especially in low-data scenarios.

Conclusions:

• Coupling image synthesis and segmentation tasks within a unified CNN framework yields superior performance compared to isolated approaches.
• The method effectively addresses the challenge of limited training data in medical image segmentation.
• This approach offers a versatile solution for cross-modality medical image synthesis and segmentation enhancement.