Interpretable Multi-Modal Image Registration Network Based on Disentangled Convolutional Sparse Coding.

Area of Science:

• Computer Vision
• Medical Imaging
• Machine Learning

Background:

• Multi-modal image registration aligns images from different sensors, which is challenging due to distinct features hindering accurate correspondence.
• Existing deep learning methods for this task often lack interpretability.

Purpose of the Study:

• To develop an interpretable deep learning framework for accurate and efficient multi-modal image registration.
• To separate image features relevant for alignment (RA) from those that are not (nRA).

Main Methods:

• A disentangled convolutional sparse coding (DCSC) model was developed to separate RA and nRA features.
• The DCSC optimization was implemented as an Interpretable Multi-modal Image Registration Network (InMIR-Net).
• An accompanying guidance network (AG-Net) was designed to supervise RA feature extraction in InMIR-Net.

Main Results:

• The InMIR-Net effectively separates RA and nRA features, improving registration accuracy and efficiency.
• The method demonstrated universal applicability to both rigid and non-rigid registration tasks.
• Experiments on diverse datasets (RGB/depth, NIR, MR, CT) confirmed the method's effectiveness.

Conclusions:

• The proposed InMIR-Net offers an interpretable and effective solution for multi-modal image registration.
• Separating alignment-relevant features significantly enhances registration performance.
• The framework is versatile, applicable to various imaging modalities and registration types.