Modeling Performance of Open Surgical Cases.

Area of Science:

• Surgical Skill Assessment
• Computer Vision in Medicine
• Medical Simulation

Background:

• Previous research utilized digital video for computer-generated surgical hand motion economy and fluidity assessments.
• This study evaluates pre-trained assessment models against expert ratings in a novel operating room (OR) setting.

Purpose of the Study:

• To test the applicability of computer vision-based surgical skill assessment models, initially developed from benchtop simulations, to in vivo suturing and tying tasks.
• To compare the performance of motion economy and fluidity of motion assessment models in a real operating room environment.

Main Methods:

• Computer vision techniques were employed to analyze video recordings of surgical suturing and tying tasks performed in the OR.
• Pre-trained assessment models for motion economy and fluidity of motion were applied to these in vivo recordings.
• Model performance was evaluated by comparing computer-generated ratings against expert surgeon ratings.

Main Results:

• Fluidity of motion assessment (R2 = 0.55) outperformed motion economy (R2 = 0.49) when compared to expert ratings.
• While 85% of ratings were within ±2 of expert scores, both models performed less effectively in the OR than on training data.
• Model accuracy was impacted by factors like changing hand postures and poor tissue contact, which were absent in training data.

Conclusions:

• Computer-generated surgical skill assessment shows potential for providing formative feedback during deliberate practice.
• Assessment variability increased in the OR compared to benchtop simulations, highlighting the need for more robust models.
• Future research should focus on incorporating diverse, bimanual video data from actual surgical environments to improve model generalizability.