Keypoint detection network for needle localization on intra-procedural MRI in MRI-guided liver interventions

Segmentation neural networks have demonstrated promising results for interventional needle localization on MRI. However, these networks require large training datasets with tedious annotation processes and additional post-processing steps that may introduce variability in the final localization results. This study aimed to develop keypoint detection networks for direct localization of the needle entry point and tip on intra-procedural liver MRI with a more efficient annotation process and more robust performance.

2D and 3D keypoint detection networks were developed by enhancing the stacked hourglass model and incorporating multi-task learning of predicting part affinity fields that connect the keypoints. The proposed networks were evaluated on intra-procedural single-slice controlled-breathing (SS-CB) and multislice controlled-breathing (MS-CB) images acquired from pre-clinical MRI-guided percutaneous liver intervention in thirteen in vivo pig subjects and compared with the results of segmentation-based UNet and Swin Transformer networks and human intra-reader variation.

The 2D and 3D keypoint detection networks achieved median needle tip and axis localization errors of 1.56 mm (1 pixel) and 1.1° for the SS-CB datasets, and 2.21 mm (~ 1.5 pixel) and 1.45° for the MS-CB datasets, respectively. Average computational times were 10 ms (2D) and 30 ms (3D). The needle localization accuracy of the keypoint networks was significantly (Wilcoxon signed-rank tests p < 0.001) higher than the UNet and Swin Transformer segmentation-based results and comparable to human intra-reader variation.

The proposed keypoint detection networks achieved rapid pixel-level needle localization on single-slice and multislice intra-procedural liver MRI with higher accuracy and a more efficient annotation process compared to segmentation-based models.