Developing the surgeon-machine interface: using a novel instance-segmentation framework for intraoperative landmark...

Area of Science:

• * Medical Artificial Intelligence
• * Surgical Computer Vision
• * Neurosurgery Technology

Background:

• * Artificial intelligence (AI) enhances surgical safety, training, and patient outcomes.
• * The Surgeon-Machine Interface (SMI) represents the intersection of surgeons and machine inference.
• * Computer vision (CV) applications in surgery are rapidly evolving.

Purpose of the Study:

• * To develop and evaluate a custom deep CV architecture for the SMI.
• * To perform instance segmentation of anatomical landmarks and surgical tools in dural arteriovenous fistula (dAVF) surgery videos.
• * To assess the performance of the SMI against state-of-the-art segmentation models.

Main Methods:

• * A custom deep convolutional neural network based on SOLOv2 architecture was developed.
• * Sparse labeling was employed, with only 133 annotated frames for training out of 8520 test frames.
• * Performance was evaluated using F1-score and mean Average Precision (mAP), comparing against MaskRCNN, YOLOv3, and SOLOv2.

Main Results:

• * The SMI achieved superior accuracy and speed, with F1-scores of 44.2% and mAP of 46.3% (at IoU > 50%).
• * Inference time was fastest at 88ms, outperforming other tested architectures.
• * The model demonstrated good generalization, identifying objects not present in the training set.

Conclusions:

• * The developed SMI platform shows significant promise for intraoperative guidance.
• * High sample efficiency and superior performance over existing models highlight its potential.
• * Future work includes transfer learning for broader surgical applications and real-time guidance capabilities.