Homography-based robust pose compensation and fusion imaging for augmented reality based endoscopic navigation system

Area of Science:

• Medical Imaging
• Surgical Technology
• Computer-Aided Surgery

Background:

• Augmented reality (AR) fusion imaging in endoscopic surgery requires precise image-to-patient registration and camera calibration.
• Independent offline calibration of these steps can lead to accumulated errors and inaccurate AR fusion.

Purpose of the Study:

• To develop a robust online calibration framework for endoscopic cameras to enhance AR fusion accuracy.
• To address the limitations of conventional independent offline calibration methods.

Main Methods:

• Proposed a 2D checkerboard-based homography estimation algorithm for local endoscopic camera pose estimation.
• Utilized the least square method and an optical tracking system to calculate a compensation matrix for real-time error correction.

Main Results:

• Reduced physical error by up to 82% and pixel error by up to 83% compared to conventional methods.
• Improved target coverage by up to 6% and demonstrated robustness to mechanical noise and endoscopic tube rotation.
• Effectively corrected fusion errors without the need for camera recalibration.

Conclusions:

• The proposed compensation method and online calibration framework are effective in improving AR fusion accuracy.
• The approach shows considerable potential for enhancing clinical practice in endoscopic surgeries.