Laparoscopic surgical skills assessment: can simulators replace experts?
Michael Pellen1, Liam Horgan, J Roger Barton
1Department of General Surgery, Northumbria Healthcare NHS Trust, Rake Lane, North Shields, UK. mikepellen@hotmail.com
Introduction:
Global Rating Scales (GRS) quantify and structure subjective expert assessment of skill. Hybrid simulators measure performance during physical laparoscopic tasks through instrument motion analysis. We assessed whether motion analysis metrics were as accurate as structured expert opinion by using GRS.
Methods:
A random sample of 10 consultant laparoscopic surgeons, 10 senior trainees, and 10 novice students were assessed on a Sharp Dissection task. Coded video footage was reviewed by two blinded assessors and scored using a Likert Scale. Correlation with metrics was tested using Spearman's rho. Inter-rater reliability was measured using intraclass correlation coefficient (ICC).
Results:
Strongest GRS-Metric correlations were found for Time/Motion/Progress with Time (Spearman's rho 0.88; p < 0.05) and Instrument Handling with Path Length (Spearman's rho 0.8; p < 0.05). Smoothness correlated with Respect for Tissue in Rater 1 (rho 0.68) but not Rater 2 (rho 0.18). Mean GRS showed stronger inter-rater agreement than individual scale components (ICC 0.68). Correlation coefficients with actual experience group were 0.58-0.74 for mean GRS score and 0.67-0.78 for metrics (Spearman's rho, p < 0.05).
Conclusions:
Metrics correlate well with GRS assessment, supporting concurrent validity. Metrics predict experience level as accurately as global rating and are construct valid. Hybrid simulators could provide resource-efficient feedback, freeing trainers to concentrate on teaching.
