Deep learning-based recognition of key anatomical structures during robot-assisted minimally invasive esophagectomy

Area of Science:

• Medical Imaging
• Artificial Intelligence in Surgery
• Surgical Robotics

Background:

• Robot-assisted minimally invasive esophagectomy (RAMIE) is complex, with high morbidity and a steep learning curve.
• Automatic anatomy recognition could enhance surgical orientation and potentially reduce complications.
• Current research on anatomy recognition in complex surgical procedures is limited.

Purpose of the Study:

• To develop a deep learning algorithm for anatomical structure recognition in thoracoscopic video frames.
• To evaluate the algorithm's accuracy and speed for real-time application in RAMIE.

Main Methods:

• Retrospective collection of 83 RAMIE procedure videos (2018-2022).
• Annotation of key anatomical structures (azygos vein/vena cava, aorta, lung) on 1050 frames by surgical experts.
• Training a convolutional neural network (CNN) on 850 frames and testing on 200 frames using Dice and 95% Hausdorff distance metrics.

Main Results:

• The algorithm achieved a median Dice score of 0.79 for azygos vein/vena cava, 0.74 for aorta, and 0.89 for lung.
• Inference time was 0.026 seconds (39 Hz), enabling real-time processing.
• Algorithm accuracy, compared to expert annotations, showed median Dice scores of 0.70 (vena cava/azygos vein), 0.88 (aorta), and 0.90 (lung).

Conclusions:

• Deep learning-based semantic segmentation demonstrates potential for anatomy recognition in RAMIE.
• The algorithm's fast inference time supports real-time clinical applicability.
• Prospective studies are needed to validate clinical utility.