Framework for Deep Learning Based Multi-Modality Image Registration of Snapshot and Pathology Images.

Area of Science:

• Medical Imaging
• Computational Pathology
• Deep Learning

Background:

• Multi-modality image registration is vital for correlating information across different medical imaging domains.
• Histopathology is the gold standard in oncologic surgery for analyzing excised tissue.
• Registering diverse imaging modalities (MRI, CT, ultrasound, white light) to pathology images presents challenges due to significant tissue deformation.

Purpose of the Study:

• To develop and validate a deep learning-based framework for multi-modality image registration.
• To address the challenge of substantial deformation in aligning microscopic pathology images with other modalities.
• To establish a robust pipeline for data acquisition, processing, and validation in medical image registration.

Main Methods:

• A deep learning framework was developed for multi-modality image registration.
• The framework was validated using ex-vivo prostate white light camera images registered to hematoxylin-eosin stained pathology images.
• A detailed pipeline covering data acquisition, pre-processing, data augmentation, loss functions, and regularization was implemented and analyzed.

Main Results:

• The proposed framework achieved a Dice Similarity Coefficient of 0.96 and a Mutual Information score of 0.54.
• A Target Registration Error of 2.4 mm and a regional Dice Similarity Coefficient of 0.70 were obtained.
• A robust training configuration was identified through comprehensive analysis of various parameters.

Conclusions:

• The developed deep learning framework enables accurate multi-modality image registration, specifically for microscopic pathology images.
• The study provides a validated pipeline and analysis of key components for successful registration, addressing significant tissue deformation.
• The findings are clinically relevant, offering improved correlation of information from different imaging modalities for applications like oncologic surgery.