MCF-SMSIS: Multi-tasking with complementary functions for stereo matching and surgical instrument segmentation

Area of Science:

• Robotics and Automation
• Computer Vision
• Medical Imaging

Background:

• Stereo matching and instrument segmentation are crucial for robotic surgery automation but are often studied in isolation.
• The interdependence between these tasks in laparoscopic surgical scenarios has been largely overlooked.

Purpose of the Study:

• To propose a novel multi-task learning framework (MCF-SMSIS) that integrates stereo matching and surgical instrument segmentation.
• To leverage instrument segmentation features to enhance stereo matching accuracy.
• To introduce new evaluation metrics (MINPD, MAXPD) for domain migration assessment in stereo matching.

Main Methods:

• Developed a multi-task framework (MCF-SMSIS) integrating stereo matching and instrument segmentation.
• Incorporated instrument segmentation features into the parallax matching block of stereo matching.
• Conducted experiments on SCARED, SERV-CT, and AutoLaparo datasets.
• Proposed MINPD and MAXPD metrics for evaluating domain migration in stereo matching.

Main Results:

• The proposed method significantly improved stereo matching accuracy, reducing EPE, >3px, and RMSE Depth by 9.5%, 12.7%, and 6.51% respectively in surgical instrument regions.
• Achieved a Dice Similarity Coefficient (DSC) of 0.9233 for instrument segmentation.
• Demonstrated efficient inference time of 0.14 seconds per image set.
• Validated effectiveness across multiple datasets.

Conclusions:

• The integrated multi-task approach (MCF-SMSIS) effectively enhances both stereo matching and instrument segmentation in laparoscopic surgery.
• Complementary features between tasks lead to improved performance and efficiency in robotic surgical automation.
• The proposed framework and metrics offer advancements for analyzing surgical video data.