Automatic generation of operation notes in endoscopic pituitary surgery videos using workflow recognition

Area of Science:

• Medical Technology
• Artificial Intelligence in Surgery
• Surgical Documentation

Background:

• Manual operation note generation is error-prone in demanding clinical settings.
• Automated systems can improve accuracy and efficiency in surgical documentation.
• Endoscopic pituitary surgery requires detailed and accurate operative notes.

Purpose of the Study:

• To develop and evaluate an automated system for generating operation notes from endoscopic pituitary surgery videos.
• To assess the feasibility of using AI for surgical step identification and documentation.
• To reduce the administrative burden on surgeons through automated note generation.

Main Methods:

• A three-stage deep learning architecture was developed for classifying surgical steps.
• Convolutional Neural Networks (CNNs) were used for frame-level binary classification.
• A discriminator and accumulator model performed video-level and multi-label step classification.
• The system was trained on 77 videos and tested on 20 videos of endoscopic pituitary surgery.

Main Results:

• The automated system achieved a weighted-F1 score of 0.80 in classifying surgical steps.
• The system successfully identified 27 predefined steps in endoscopic pituitary surgery videos.
• Classifications were integrated into a template and enriched with video analytics.

Conclusions:

• Automatic generation of operation notes from surgical videos is feasible.
• This technology can assist surgeons by improving accuracy and efficiency in documentation.
• AI-powered systems hold promise for enhancing patient care through better record-keeping.