Next-generation surgical navigation: Marker-less multi-view 6DoF pose estimation of surgical instruments

Area of Science:

• Computer-assisted surgery
• Surgical robotics
• Medical image analysis

Background:

• Traditional computer vision is increasingly used in surgery, with a focus on replacing marker-based tracking for instrument localization.
• Current single-view pose estimation methods lack the accuracy needed for surgical navigation.
• Marker-less 6DoF (six degrees of freedom) pose estimation is a key area of research in computer-assisted surgery.

Purpose of the Study:

• Investigate the benefits of multi-view camera setups for accurate and occlusion-robust 6DoF pose estimation of surgical instruments.
• Derive recommendations for an ideal camera system to address operating room challenges.
• Evaluate the feasibility of marker-less surgical instrument tracking as an alternative to marker-based systems.

Main Methods:

• Developed a multi-view RGB-D video dataset of ex-vivo spine surgeries with static and head-mounted cameras.
• Collected rich annotations for surgeon, instruments, and patient anatomy.
• Evaluated state-of-the-art single-view and multi-view pose estimation methods, analyzing camera configurations and data limitations.

Main Results:

• Achieved average position errors of 1.01mm and orientation errors of 0.89° for a surgical drill.
• Achieved average position errors of 2.79mm and orientation errors of 3.33° for a screwdriver under optimal conditions.
• Demonstrated that multi-view systems enhance pose accuracy, occlusion robustness, and generalizability compared to single-view methods.

Conclusions:

• Multi-view setups offer significant advantages for marker-less surgical instrument tracking.
• The proposed multi-camera system design shows promise for clinical application.
• Marker-less tracking is emerging as a practical alternative to current marker-based systems in surgery.