Using Contact Forces and Robot Arm Accelerations to Automatically Rate Surgeon Skill at Peg Transfer

Area of Science:

• Robotics
• Surgical Education
• Machine Learning

Background:

• Robotic minimally invasive surgery training often relies on subjective, time-consuming manual skill assessments by expert surgeons.
• Standardized rating scales for evaluating trainee performance on inanimate tasks can be tedious and prone to subjectivity.

Purpose of the Study:

• To develop and validate an automated system for evaluating surgical skill in robotic surgery trainees.
• To analyze contact force, instrument acceleration, and task completion time for objective skill assessment.

Main Methods:

• Thirty-eight participants with varying robotic surgery experience performed peg transfer tasks using a da Vinci robot.
• A Smart Task Board recorded force, acceleration, and time data, serving as input for machine learning algorithms.
• Human raters provided ground-truth Global Evaluative Assessment of Robotic Skill (GEARS) scores for algorithm training and validation.

Main Results:

• Machine learning models achieved good to excellent agreement with human expert ratings for surgical skill assessment.
• Regression models demonstrated higher accuracy and efficiency in predicting GEARS scores compared to classification models.
• Effective skill rating was achieved even without incorporating contact force data.

Conclusions:

• Objective and reliable assessment of robotic surgery skills, specifically peg transfer, is achievable using regression models trained on sensor data.
• The developed system utilizes external sensors to gather force, acceleration, and time features for skill evaluation.
• This automated approach is expected to improve the learning experience for surgical trainees during inanimate task practice.