Temporal variability of surgical technical skill perception in real robotic surgery

Area of Science:

• Robotic Surgery
• Surgical Skill Assessment
• Medical Performance Metrics

Background:

• Current methods for reporting surgeon technical skill rely on summary scores from non-experts, faculty surgeons, or automated systems.
• These established metrics may not fully capture the dynamic nature of surgical performance during lengthy procedures.

Purpose of the Study:

• To investigate whether significant fluctuations in technical skill assessments occur within long surgical video durations.
• To determine if surgeon technical skill varies measurably over the course of a robotic surgery.

Main Methods:

• Analyzed 12 robotic surgery videos (12-15 minutes each) by evaluating perceived technical skill at one-minute intervals.
• Employed linear mixed-effects models to compare minute-by-minute ratings, accounting for inter- and intra-rater variability.

Main Results:

• Measurable changes in technical skill over time were detected in 11 out of 12 surgeons studied.
• The modeling of skill score changes over time significantly contributed to prediction models, confirming temporal variations.

Conclusions:

• Findings challenge the adequacy of current evaluation durations for accurately rating surgical technical skill in longer procedures.
• This variability suggests potential label noise in supervised machine learning models for skill assessment.
• Future evaluations may require reporting surgeon skill variability alongside mean scores for a comprehensive understanding of performance.