Machine Learning based Classification of Local Robotic Surgical Skills in a Training Tasks Set.

Area of Science:

• Robotics
• Surgical Training
• Machine Learning

Background:

• Objective performance measurement is crucial for surgical skill development but remains a challenge in current training systems.
• Existing methods often provide only global performance metrics post-task completion, limiting granular feedback.
• Developing objective, localized performance evaluation is key to improving surgical training efficacy.

Purpose of the Study:

• To propose and validate a temporal evaluation scheme for assessing local surgical performance during training tasks.
• To automatically classify surgeons as expert or non-expert based on their performance across different time intervals.
• To provide a quantitative tool for visualizing skill improvement opportunities in surgical training.

Main Methods:

• A temporal evaluation scheme was developed to analyze performance in discrete time intervals during surgical tasks (knot-tying, needle-passing, suturing).
• Three machine learning classifiers—K-Nearest Neighbors, Random Forest, and Support Vector Machine—were employed for surgeon classification.
• The system aimed to classify surgeons based on performance segments without requiring segment-level data labeling.

Main Results:

• The proposed method achieved high classification performance, with accuracy ranging from 83% to 100%.
• Area Under the Receiver Operating Characteristic Curve (AUC-ROC) and F1-Score both ranged from 88% to 100%.
• The Support Vector Machine classifier demonstrated the highest performance in distinguishing between expert and non-expert surgeons.

Conclusions:

• The proposed temporal evaluation scheme offers a novel approach to assessing local surgical performance during training.
• This method can be integrated into surgical trainers to provide quantitative feedback on skill development.
• The findings suggest a valuable tool for enhancing the learning process and identifying areas for surgical skill improvement.