Enhancing ultrasound augmentation of laparoscopic video using hybrid electromagnetic-ArUco tracking

This work aims to develop and validate a hybrid electromagnetic(EM)-ArUco tracking framework integrated into a laparoscopic ultrasound augmentation system, aiming to improve accuracy, stability, and robustness in intraoperative visualization.

A novel hybrid tracking method was developed that combines continuous hardware-based EM tracking with accurate but intermittent computer vision-based ArUco marker tracking, utilizing dynamically computed correction matrices to optimize the real-time spatial alignment of ultrasound and laparoscopic images. Validation involved three experiments: image-based analysis measuring pixel error between projected and ground-truth needle tips, target registration error analysis quantifying the distance between an EM- tracked stylus tip and the inverse projection from the ultrasound, and an ex vivo tissue experiment to assess distortions in a simulated clinical environment.

Hybrid tracking demonstrated a significant reduction in tracking error compared to EM tracking alone. Hybrid tracking reduced projection error from 116 ± 49 px to 49 ± 18 px and decreased registration error from 2.1 ± 0.9 mm to 1.1 ± 0.6 mm. Under the influence of externally applied distortion, hybrid tracking improved tracking accuracy by approximately 62.3%. The system also exhibited robustness to camera rotation and zoom changes during periodic ArUco marker detection.

The hybrid tracking framework significantly improved accuracy, stability, and robustness compared to EM tracking. This integration has the potential to enhance intraoperative visualization, thereby contributing to safer and more effective minimally invasive surgeries.