An Indirect Multimodal Image Registration and Completion Method Guided by Image Synthesis

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Multimodal registration of CT and MRI is crucial for clinical diagnosis.
• Significant variations between modalities pose challenges for accurate registration.
• MRI and CT provide complementary information, necessitating effective integration methods.

Purpose of the Study:

• To develop an indirect multimodal image registration method using synthetic CT (sCT).
• To design a deep learning-based generative network for high-quality sCT synthesis.
• To address the challenge of registering images from different modalities by transforming it into a monomodal registration task.

Main Methods:

• Proposed a Conditional Auto-Encoder Generative Adversarial Network (CAE-GAN) combining Variational Auto-Encoder (VAE) and Generative Adversarial Network (GAN) principles.
• Utilized multicontrast MRI to generate synthetic CT (sCT) as an intermediary for registration.
• Developed a method for repairing abnormal MRI while registering it to normal CT, guided by CT images.

Main Results:

• CAE-GAN produced high-quality synthetic CT images, even with limited training data.
• The indirect sCT-guided registration significantly reduced the complexity of multimodal MRI-CT registration.
• The method successfully repaired abnormal MRI regions during registration to normal CT.

Conclusions:

• The proposed CAE-GAN offers an effective solution for synthetic CT generation in medical imaging.
• Indirect registration via sCT simplifies and improves the accuracy of multimodal image alignment.
• This approach enhances the utility of combining CT and MRI data for clinical analysis and diagnosis.