Calibration-free 3D-2D surface registration for image guided intervention

Accurate and robust three-dimensional 3D to two-dimensional 2D registration is crucial for augmented reality and image guidance in minimally invasive surgery. Conventional methods require precise camera calibration, which is challenging to achieve, especially for cameras with high optical zoom, significantly impacting the accuracy of 3D-2D registration algorithms. This paper proposes a novel method for 3D-2D surface registration that eliminates the need for camera calibration. Under small field of view, the method transforms the 3D-2D registration into an equivalent 2.5D-2.5D process, simplifying registration and improving efficiency. Moreover, by introducing a bias-correction technique to pseudo depth map estimation, the multimodal registration problem is converted into an approximate unimodal registration problem, enhancing robustness across various initial conditions and camera types. The method is validated through experiments on both a high-zoom surgical exoscope dataset with unknown camera parameters and a benchmark endoscopic dataset with known camera parameters. For exoscope experiments, the proposed method achieves a highly accurate registration with mean target registration error of 1.56 mm for rigid phantoms and 1.53 mm for deformable cadaver brains. In endoscope experiments, it achieves a 2D fiducial distance error of 1.95 mm, demonstrating efficacy across different scenarios and camera models.