Simulation-to-real domain adaptation with teacher-student learning for endoscopic instrument segmentation

Area of Science:

• Medical image analysis
• Computer-assisted surgery
• Machine learning in healthcare

Background:

• Automated surgical scene understanding requires accurate segmentation of surgical instruments in endoscopic videos.
• Fully supervised deep learning methods are hindered by the time-consuming manual annotation process by clinical experts.

Purpose of the Study:

• To develop a teacher-student learning approach for unsupervised domain adaptation in endoscopic image segmentation.
• To address the challenge of segmenting surgical instruments using limited annotated data by leveraging both simulated and real endoscopic video streams.

Main Methods:

• A teacher-student learning framework was introduced, jointly learning from annotated simulation data and unlabeled real endoscopic data.
• The method focuses on unsupervised domain adaptation to bridge the gap between simulation and real-world endoscopic images.

Main Results:

• The proposed framework demonstrated superior performance compared to existing methods on three endoscopic instrument segmentation datasets.
• Analysis identified key factors influencing performance, revealing the strengths and limitations of the approach.

Conclusions:

• The approach effectively utilizes unlabeled real endoscopic video frames to enhance generalization performance.
• This work advances the goal of effective surgical instrument segmentation in settings with limited annotations, moving closer to practical applications.