Cross Domain Self-Prompting SAM2 for Intraoperative OCT Video Segmentation

Area of Science:

• Ophthalmology
• Medical Imaging
• Computer Vision

Background:

• Retinal procedures demand high precision to prevent iatrogenic injuries.
• Intraoperative Optical Coherence Tomography (iOCT) provides real-time visualization of tissue-instrument interactions.
• Accurate iOCT segmentation is crucial for automated distance estimation and safety alerts in vitreoretinal surgery.

Purpose of the Study:

• To develop a real-time iOCT video segmentation framework using a cross-domain self-prompting SAM2 approach.
• To enable automated distance estimation between surgical instruments and retinal surfaces for enhanced intraoperative safety.
• To overcome limitations of frame-wise segmentation and reliance on simulated datasets.

Main Methods:

• Proposed a cross-domain self-prompting SAM2 framework for 50 FPS iOCT video segmentation.
• Introduced a pseudo mask generator for knowledge transfer from clinical OCT to iOCT datasets.
• Enabled fully automatic video inference without manual prompts.

Main Results:

• Achieved mean Dice scores of 91.21% and 89.16% on two real vitreoretinal surgery iOCT datasets.
• Outperformed the second-best method by significant margins (4.47% and 0.48%).
• Estimated tool-to-retina distance with a mean error of 17μm, below average human hand tremor.

Conclusions:

• The proposed framework significantly advances real-time iOCT video segmentation.
• Demonstrated potential for enhancing intraoperative safety through accurate distance estimation.
• This is the first SAM2-based framework for fully automatic segmentation of real vitreoretinal iOCT videos without manual prompting.