Predicting future surgical steps during MCA aneurysm clipping using a multimodal transformer

Intracranial aneurysm clipping is technically demanding, with dynamic anatomy and evolving intraoperative decisions. Although AI has been applied to retrospective surgical phase recognition, near-future surgical step prediction remains largely unexplored in neurosurgery. This study evaluated the feasibility of fixed-horizon surgical step prediction in recorded microscope videos of middle cerebral artery aneurysm clipping operations.

We retrospectively analyzed 25 uncomplicated MCA bifurcation aneurysm clipping surgeries by a single neurosurgeon, using 18 for training and 7 for independent testing. Cases were annotated into 12 standardized operative steps. A transformer-based prediction framework was evaluated with three input configurations: video features alone, prior human-annotated step labels alone, and combined video plus step-label inputs. With a sliding-window approach, each configuration used 1 min of input data to predict the operative step labels occurring during the subsequent 1 min. Performance was assessed by accuracy, weighted F1 score, and sequence-level agreement with ground truth.

The multimodal model achieved the highest mean accuracy and weighted F1 score, 0.683 and 0.673, compared with 0.606 and 0.577 for the annotation-only model and 0.477 and 0.447 for the video-only model. The multimodal model also showed the best sequence-level alignment, with a normalized edit distance of 0.430 and edit score of 0.570.

Fixed-horizon surgical step prediction during MCA aneurysm clipping was feasible under controlled input conditions. Multimodal modeling provided the strongest predictive performance. These findings represent upper-bound performance and require validation in fully automated recognition-to-prediction pipelines.