SPLNet: Self-Distilled Progressive Learning Network for Deformable Medical Image Registration

Deformable medical image registration serves as a vital prerequisite for multimodal image fusion, disease diagnosis, and surgical navigation. Although existing learning-based registration methods are very popular, they mainly rely on the pyramid and cascaded structure-based networks trained in an end-to-end manner, which may lead to suboptimal parameter optimization and limited accuracy. In this work, we have proposed a novel self-distilled progressive learning network (SPLNet) utilizing progressive learning and knowledge distillation (KD). In the proposed SPLNet, the optimization order of each component is reasonably arranged based on the idea of evolution and the corresponding learning gaps are filled by using the self-distillation (SD) scheme. The layers of different levels in SPLNet are progressively updated based on the depth-wise pyramid weight-inherited evolution, thereby enhancing the sensitivity of SPLNet to multiscale deformations. Meanwhile, the feature maps of trained layers are treated as teachers and shifted to distill the learning process of newly generated layers. Besides, more subnetworks in SPLNet are progressively appended based on the width-wise cascaded evolution, enabling SPLNet to learn the decomposed deformations more easily. Distinctively, a distribution-consistent distillation is proposed to fill the learning gaps between the newly appended subnetworks and old ones. Indeed, the introduction of two types of evolution and distillation enables SPLNet to handle complex deformations more effectively. Experiments on both mono-modal datasets (LPBA40, OASIS, SLIVER, and LSPIG) and a multimodal dataset (MMWHS) demonstrate the superior registration performance of SPLNet to several baseline methods in terms of Dice score, 95% maximum Hausdorff distance, folds, and inference speed. The code is available at https://github.com/YiboWang3813/SEN.