AI-Based Automated Instrument Tracking During Endoscopic Skull Base Surgery: A Pilot Feasibility Study

Endoscopic endonasal approaches (EEA) to skull base tumors are a technically challenging procedure for both surgeons and trainees. Artificial intelligence and computer vision (CV) may offer an objective analysis that provides surgical assessment on technique and enhanced educational feedback for surgeons of all levels.

Patients undergoing EEA from a tertiary care center between January 2022 and January 2023 were retrospectively analyzed. A large, publicly available CV model fine-tuned on endoscopic surgical video in collaboration with Surgical Data Science Collective (SDSC) was used for frame-by-frame detection of 14 different surgical instruments. Tool usage was depicted across different surgical phases of EEA. Dice-Sørensen Similarity Coefficients (DICE) were used to identify similarities between tool motion patterns across patients.

Ten patients (5 male, 5 female, ages 52.1 ± 13.8 years) were included, with over 35 h of video footage analyzed encompassing over 1.9 million individual frames. The sellar phase accounted for the majority of the analyzed frames. Within the sellar phase, the most frequently identified tools included suction (661,785), drill (138,031), Rhoton dissector (116,823), and cotton patties (105,038). Overall, DICE with higher frequency use exhibited substantially increased DICE scores: 0.52 for drill, 0.63 for Rhoton dissector, 0.67 for cotton patties, and 0.74 for suction.

CV models reliably identified surgical instruments and can be used to capture both high-frequency patterns and deviations from common patterns in instrument usage across cases of endoscopic surgery.