Mapping surgical fields by moving a laser-scanning multimodal scope attached to a robot arm

Area of Science:

• Medical Imaging
• Robotics
• Computer Vision

Background:

• Brain tumor removal faces visualization challenges due to indistinguishable tumor and healthy tissues.
• Fluorescence imaging aids tumor delineation but hinders reflectance-based anatomical visualization.

Purpose of the Study:

• To develop an advanced endoscopic system for improved brain tumor visualization and resection.
• To create high-resolution, multimodal 3D surface maps of the surgical field.

Main Methods:

• An ultrathin/flexible scanning fiber endoscope (SFE) acquiring simultaneous reflectance and fluorescence images.
• A robotic arm providing programmable motion for SFE scanning.
• SIFT feature matching and bundle adjustment for 3D reconstruction of multimodal surface maps.

Main Results:

• Generated multimodal 3D surface maps of surgical phantoms with fluorescence-guided tumor spots.
• Demonstrated improved visualization and potential for enhanced robotic path planning.
• Reduced 3D map generation time by one third using pre-programmed robotic trajectories.

Conclusions:

• The developed SFE system and multimodal 3D mapping enhance surgical visualization in brain tumor resection.
• Optimized computer vision algorithms and robotic motion significantly improve the efficiency of map generation.
• This technology offers a promising advancement for image-guided neurosurgery and robotic path planning.