Toward real-time alignment of 3D CT and 2D X-ray with multi-stage CNNs

2D X-ray imaging is widely employed in image-guided interventions to provide real-time visualization of patient anatomy and interventional devices. However, lacking depth information and soft-tissue contrast, intraoperative X-rays are often complemented with high-resolution preoperative 3D computed tomography (CT) scans. Clinicians must mentally register the 3D preoperative information onto the 2D visualization, increasing cognitive workload and motivating the need for automated solutions. Traditionally, this registration task is formulated as estimating the position of the X-ray source relatively to the CT scan. State-of-the-art 3D/2D registration approaches are trained using synthetic CT-to-X-ray projections, but these methods still require manual annotations and employ a time-consuming optimization that limits their deployment for live image guidance. In this paper, we propose LXPose (live X-ray pose estimation), a fast multi-stage 3D/2D registration framework for real-time image guidance. Specifically, we introduce an efficient two-stage CNN that effectively bypasses slow optimization for fast inference. Critically, we also eliminate the need for any manual annotations by introducing an automated strategy for landmark extraction. Finally, we train LXPose using a projection loss for high accuracy, and apply extensive data augmentation in a first attempt to reduce the domain gap between the synthetic training X-rays and real testing data. We demonstrate LXPose on two datasets from different anatomical regions, where it yields results comparable to the state-of-the-art, while reducing inference time by two orders of magnitude, from several seconds to 20 ms. Overall, our results demonstrate the potential of LXPose for real-time clinical deployment. Our code is available at https://github.com/fedefacente/LXPose.