Explainable machine learning for automated robotic surgical skill assessment using rich kinematic features

Area of Science:

• Robotics and Artificial Intelligence in Medicine
• Surgical Skill Assessment
• Machine Learning for Healthcare

Background:

• Robotic-assisted surgery (RAS) enhances minimally invasive procedures but requires objective skill assessment for training.
• Current methods often lack subject-independent validation and transparent explanations.
• Assessing surgical skills from console kinematics is crucial for competency-based training.

Purpose of the Study:

• To develop and validate an explainable machine learning framework for automated skill classification in RAS using kinematic data.
• To systematically compare various classifiers and feature extraction methods for skill assessment.
• To provide transparent, subject-independent evaluation of surgical skills.

Main Methods:

• Extracted 633 kinematic features from 103 JIGSAWS trials across three tasks.
• Compared six supervised classifiers using leave-one-subject-out cross-validation (LOSO-CV).
• Utilized Random Forest (RF) with SHapley Additive exPlanations (SHAP) for model interpretability.

Main Results:

• The RF model achieved 97.1% accuracy in skill classification, outperforming PCA-reduced features (92.2%) and segment-level modeling (88.3%).
• SHAP analysis identified jerk and velocity variability as key discriminators of surgical skill.
• High in-task performance (97-100%) was observed, with reduced off-task transfer (61-95%).

Conclusions:

• An explainable AI framework using kinematic data enables accurate and transparent assessment of surgical skills.
• Jerk and velocity variability are critical kinematic indicators for skill differentiation.
• Further multi-center validation is needed for clinical deployment of these training analytics.