3-D Tracking for Augmented Reality Using Combined Region and Dense Cues in Endoscopic Surgery

An augmented reality (AR) technique has recently gained its popularity in minimally invasive surgery. Tracking is a crucial step to achieve precise AR. Besides optical tracking in traditional medical AR, visual tracking attracts a lot of attention due to its generality. Moreover, when the target organ's 3-D model can be obtained from preoperative images and under the model rigidity assumption, tracking is then converted into a problem of computing the six-degree-of-freedom pose of the 3-D model. In this paper, we introduce a robust tracking algorithm in our endoscopic AR system, where we combine the benefits of both region and dense cues in a unified framework. Each kind of cues alone may not be adequate for tracking in endoscopic surgery. However, they have complementary characteristics, with region cues being more robust to motion blur and fast motion, and dense cues being more accurate when motion is not large. We also propose an appearance model adaption method and an occlusion processing method to effectively handle occlusions. Experiments on both synthetic dataset and simulated surgical environment show the effectiveness and robustness of our proposed method. This work presents a novel tracking strategy in medical AR applications.