Segmenting and classifying activities in robot-assisted surgery with recurrent neural networks

Area of Science:

• Robotics and Automation
• Medical Education Technology
• Machine Learning in Healthcare

Background:

• Automated surgical activity recognition is crucial for objective training assessment.
• Previous methods focused on simple gestures, neglecting complex maneuvers.
• Maneuvers align better with surgical training curricula despite their complexity.

Purpose of the Study:

• To develop and evaluate recurrent neural networks (RNNs) for automated surgical maneuver recognition.
• To compare different RNN architectures for their effectiveness in classifying surgical activities.
• To assess the feasibility of using RNNs for higher-granularity surgical feedback.

Main Methods:

• Four RNN architectures (simple RNN, LSTM, GRU, mixed history RNN) were implemented.
• Kinematic data from surgical procedures were used for training and evaluation.
• Performance was measured using error rate and normalized edit distance.
• Hyperparameter sensitivity was analyzed using functional ANOVA.

Main Results:

• State-of-the-art performance was achieved for maneuver recognition (4 maneuvers) and gesture recognition (10 gestures).
• Error rates and normalized edit distances were significantly improved compared to prior methods.
• Specific RNN architectures and hyperparameters were identified as critical for performance.

Conclusions:

• Automated surgical maneuver recognition using RNNs is feasible and effective.
• This technology can provide targeted assessment and feedback at a higher level of detail for surgical trainees.
• The hyperparameter analysis provides valuable insights for future RNN-based activity recognition research.