Deep learning-based head and neck deformable image registration using spatio-temporal analysis and self attention

Significant anatomical changes during head and neck cancer (HNC) radiotherapy challenge accurate dose delivery. Deformable image registration (DIR) is essential for adaptive radiotherapy (ART), yet conventional methods are too slow for online clinical use. This study proposed a novel deep learning-based DIR algorithm for longitudinal HNC imaging.

We used sixty HNC patient datasets, each containing a planning CT (pCT) and six weekly cone-beam CTs (CBCTs). Fifty datasets were used for training with cross-validation, and the remaining ten were reserved for testing. The proposed DIR algorithm is a patch-based model that integrates 3D convolutional neural networks, self-attention, and a convolutional Long Short Term Memory to model temporal deformations. The model predicted bidirectional deformation vector fields and was trained with a composite loss function combining image similarity, DVF smoothness, and inverse consistency. Performance was benchmarked against the large deformation diffeomorphic metric mapping (LDDMM) algorithm using Dice similarity coefficient (DSC), Hausdorff distance, and Jacobian analysis.

The proposed method achieved significantly faster inference, performing bidirectional DIR between the pCT and all six weekly CBCTs in under 3 min and averaging about 30 s per patient, while matching or exceeding LDDMM's accuracy. DSC remained above 0.8 for all key structures, and the method demonstrated improved DVF consistency with lower mean and 95th percentile Hausdorff distances. Unlike LDDMM, it required no manual parameter tuning, providing consistent results.

The proposed DIR algorithm enabled rapid, accurate, and consistent image registration, supporting real-time ART workflows and retrospective dose accumulation in personalized radiotherapy.