Cross-Modality Image Registration Via Generating Aligned Image Using Reference-Augmented Framework.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Cross-modality image alignment (e.g., MR-CT) is crucial in medical applications but faces limitations with traditional registration or Image-to-Image (I2I) translation.
• Existing methods struggle with preserving image details and achieving accurate structural alignment.

Purpose of the Study:

• To introduce a novel deep learning framework, "Register by Generation (RbG)", for accurate cross-modality image alignment.
• To develop a method that preserves the intensity and contrast of the reference image while ensuring structural alignment.
• To enable self-supervised training on misaligned datasets, removing the need for pre-aligned data.

Main Methods:

• A two-stage 2D deep learning approach: 1) Semi-global reference-augmented image synthesis using Patch Adaptive Instance Normalization (PAdaIN) for initial alignment and hallucination reduction.
• 2) Detailed refining network with a Deformation-Aware Cross-Attention (DACA) block for recovering fine details and texture transfer.
• Novel combination of loss functions for self-supervised training on misaligned datasets.

Main Results:

• RbG demonstrates superior performance in structural alignment and distributional consistency across multiple misaligned datasets compared to conventional methods.
• The method shows robustness when tested against simulated intentional misalignments.
• Experiments highlight the broad applicability of RbG in case studies and downstream medical image segmentation tasks.

Conclusions:

• The "Register by Generation (RbG)" framework offers a significant advancement in cross-modality image alignment.
• RbG effectively addresses limitations of existing methods by preserving image details and enabling self-supervised learning.
• The approach shows promise for various medical imaging applications, including segmentation.