Multimodal 3D medical image registration guided by shape encoder-decoder networks.

Area of Science:

• Medical image analysis
• Computer vision
• Machine learning

Background:

• Nonlinear multimodal image registration, such as fusing computed tomography (CT) and magnetic resonance imaging (MRI), is crucial for medical applications.
• Existing methods often rely on global or local similarity measures, which can lead to local optima.
• Many deep learning methods require strongly supervised, aligned image pairs, limiting practical use.

Purpose of the Study:

• To develop a novel, weakly supervised approach for nonlinear multimodal image registration.
• To overcome limitations of existing methods by utilizing anatomical shape information.
• To enable practical applications by reducing reliance on strongly supervised data.

Main Methods:

• A shape-constrained encoder-decoder segmentation network is jointly trained on individually segmented CT and MRI data.
• An iterative energy-based minimization scheme leverages intermediate nonlinear shape representations generated by the network.
• The method exploits anatomical shape information, requiring only segmentation labels for each modality.

Main Results:

• The proposed approach robustly and accurately aligns 3D scans from a multimodal whole-heart segmentation dataset.
• Performance surpasses that of classical unsupervised registration frameworks.
• The method is easily integrated into deep learning frameworks and executable on GPUs due to its reliance on gradient optimization.

Conclusions:

• A novel, integrated approach for weakly supervised multimodal image registration is presented.
• Exploration of intermediate shape features as registration guidance yields promising results.
• The findings encourage further research into shape-informed registration techniques.