Using Surgeon Hand Motions to Predict Surgical Maneuvers

Area of Science:

• Surgical simulation and machine learning applications.
• Computer vision in medical training.

Background:

• Objective assessment of surgical skills is crucial for training.
• Automated recognition of surgical maneuvers can streamline video review.
• Surgical maneuver classification aids in skill evaluation.

Purpose of the Study:

• To explore machine learning (ML) techniques for predicting surgical maneuvers from video.
• To assess the accuracy of ML models in classifying suturing, tying, and transition states.
• To determine the effectiveness of different ML approaches for surgical video analysis.

Main Methods:

• Collected video data of 37 clinicians performing suturing simulations.
• Applied decision trees, random forests, and hidden Markov models to classify maneuvers.
• Classified surgical maneuvers every 2 seconds (60 frames) of video data.

Main Results:

• Random forest models achieved 74% accuracy in classifying surgical maneuvers.
• Hidden Markov model adjustments improved random forest predictions to 79% for specific suturing types.
• Training models across all users enhanced prediction accuracy by 10%.

Conclusions:

• Random forest with hidden Markov modeling offers the best prediction of surgical maneuvers.
• Marker-less video analysis shows comparable accuracy to robot-assisted platforms.
• This technology can enhance surgical procedure review for training and coaching.