macJNet: weakly-supervised multimodal image deformable registration using joint learning framework and multi-sampling...

Area of Science:

• Medical Image Analysis
• Computer Vision
• Radiology

Background:

• Deformable multimodal image registration is crucial for medical image analysis but challenging due to intensity distortion and large deformations.
• Accurate dense correspondences between images from different modalities (e.g., CT and MRI) are difficult to establish.
• Existing methods struggle with significant cross-modal variations and complex anatomical structures.

Purpose of the Study:

• To introduce macJNet, a weakly-supervised method for accurate deformable multimodal medical image registration.
• To develop a joint learning framework integrating registration and segmentation networks.
• To propose a novel modality-independent neighborhood descriptor (macMIND) for enhanced feature representation.

Main Methods:

• macJNet employs a joint learning framework with a registration network and two semi-supervised segmentation networks.
• A multi-sampling cascaded modality independent neighborhood descriptor (macMIND) captures self-similarity context across orientations and scales.
• The framework leverages segmentation networks for semantic correspondence and improves segmentation through registration consistency.

Main Results:

• macJNet demonstrates superior performance in multimodal medical image registration compared to state-of-the-art methods.
• The proposed macMIND descriptor effectively enhances cross-modal feature representation for registration.
• The joint learning framework improves both registration accuracy and segmentation performance.

Conclusions:

• macJNet offers a robust and effective solution for deformable multimodal medical image registration.
• The macMIND descriptor and joint learning framework significantly advance the field of cross-modal medical image analysis.
• This method holds promise for improving diagnostic accuracy and treatment planning in clinical settings.