End-to-end multimodal image registration via reinforcement learning

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Multimodal image registration is crucial for integrating complementary information from different medical imaging modalities.
• Accurate correspondence finding is challenging due to variations in image characteristics across modalities.
• Existing convolutional methods often require custom-designed features and similarity metrics, limiting adaptability.

Purpose of the Study:

• To develop a robust and adaptable multimodal image registration method.
• To address the limitations of custom-designed components in current convolutional registration techniques.
• To improve the accuracy and efficiency of medical image registration.

Main Methods:

• Image registration is framed as a decision-making problem solved by an artificial agent.
• Asynchronous reinforcement learning trains the agent, incorporating convolutional long-short-term-memory for spatial-temporal feature extraction.
• A landmark error-driven reward function and Monte Carlo rollout strategy enhance registration accuracy.

Main Results:

• The proposed method achieves state-of-the-art performance in multimodal medical image registration.
• Experiments on paired CT and MR images of nasopharyngeal carcinoma patients validate the approach.
• Implicit learning of features and similarity metrics demonstrates superior adaptability.

Conclusions:

• Reinforcement learning offers a powerful framework for multimodal image registration.
• The developed method provides a more generalized and accurate solution compared to traditional approaches.
• This technique holds significant potential for advancing medical image analysis applications.