Stereoscopic vision-based robotic manipulator extraction method for enhanced soft tissue reconstruction

The availability of digital stereoscopic video feedback on surgical robotic platforms allows for a variety of enhancements through the application of computer vision. Several of these enhancements, such as augmented reality and semi-automated surgery, benefit significantly from identification of the robotic manipulators within the field of view. A method is presented for the extraction of robotic manipulators from stereoscopic views of the operating field that uses a combination of marker tracking, inverse kinematics, and computer rendering. This method is shown to accurately identify the locations of the manipulators within the views. It is further demonstrated that this method can be used to enhance 3D reconstruction of the operating field and produce augmented views.