Weakly-supervised convolutional neural networks for multimodal image registration.

Area of Science:

• Medical Imaging
• Computer Vision
• Machine Learning

Background:

• Supervised learning for multimodal image registration faces challenges due to the lack of ground-truth voxel-level spatial correspondence.
• Anatomical labels offer a more reliable and practical alternative for obtaining correspondence information.

Purpose of the Study:

• To develop a method for inferring voxel-level transformation from higher-level anatomical label correspondence.
• To create an end-to-end convolutional neural network for predicting displacement fields to align labelled structures.

Main Methods:

• An end-to-end convolutional neural network was designed to predict displacement fields for aligning multimodal images using anatomical labels during training.
• The network utilizes unlabelled image pairs for inference, enabling automated and real-time 3D deformable image registration.
• Network architecture variants were evaluated for registering T2-weighted MRI and 3D transrectal ultrasound images in prostate cancer patients.

Main Results:

• The proposed method achieved a median target registration error of 3.6 mm on landmark centroids.
• A median Dice similarity coefficient of 0.87 was obtained for prostate glands.
• Cross-validation experiments on 108 image pairs from 76 patients demonstrated high-quality registration.

Conclusions:

• The developed strategy effectively infers voxel-level transformations from anatomical labels, overcoming the ground-truth data limitation in supervised learning for image registration.
• The real-time, automated registration algorithm is versatile, utilizing diverse anatomical labels and performing without initialisation or labels at inference.