External validation of a motion capture-based surgical skill assessment system in laparoscopic simulation training...

Area of Science:

• Surgical Education Technology
• Medical Simulation
• Machine Learning in Healthcare

Background:

• Objective assessment of surgical skills is crucial for effective training.
• Current simulation environments often lack comprehensive, real-time feedback mechanisms.
• Motion capture (Mocap) metrics offer potential for detailed analysis of surgical movements.

Purpose of the Study:

• To externally validate a surgical skill assessment system providing real-time feedback.
• To assess the system's ability to classify skill levels and predict Global Operative Assessment of Laparoscopic Skills (GOALS) scores.
• To evaluate participant satisfaction with the Mocap-based feedback system.

Main Methods:

• Participants performed laparoscopic dissection and suturing tasks on porcine models.
• Motion capture technology recorded surgical instrument movements.
• A machine learning (ML) algorithm provided real-time, quantitative feedback and predicted skill levels and GOALS scores.

Main Results:

• The system achieved classification accuracies of 67.3% (Dissection) and 56.9% (Suturing).
• Correlation coefficients for predicted vs. expert GOALS scores were 0.86 (Dissection) and 0.91 (Suturing).
• 88% of participants found the feedback easy to understand, with 75% satisfied with the system.

Conclusions:

• The validated system offers reliable, real-time feedback for surgical simulation training.
• The ML algorithm effectively predicts surgical skill levels and GOALS scores.
• This technology can enhance objective performance assessment and accelerate skill acquisition.