Development of a web-based laparoscopic technical skills assessment and testing instrument: a pilot study

Current surgical training programs rely heavily on subjective assessments to measure operative proficiency, despite heavy emphasis on standardized testing as a means to rank scholastic ability. A compact laparoscopic simulator was developed with the intention to create a technical skill evaluation system that resembles standardized testing to provide the user with real-time percentile scores in a variety of skill metrics. The calculation of percentiles is only accurate if the pool of scores resembles a specific distribution (e.g., normal or log-normal distribution). We hypothesize that the grading measures provided by the simulator will follow normal or log-normal distributions.

A total of 29 surgical trainees with varying levels of laparoscopic experience were surveyed regarding their current training, proficiencies, and experience with the Fundamentals of Laparoscopic Surgery curriculum and then asked to perform a standard peg-transfer task 5 times. A proprietary device placed along the trocars of a laparoscopic box trainer was used to gather data that, when subjected to unique algorithms, gave real-time, web-based feedback to trainees on the following metrics: volume of instrument use, economy of movement, angular instrument path, instrument rotation, bimanual coordination, smoothness, time to task completion, and depth perception. Numerical data were plotted on a frequency histogram. Minitab software was used to identify if individual metrics fit a standard distribution curve. Analysis of variance was used to differentiate among 3 established physician skill levels, as a means of assessing construct validity.

In the goodness-of-fit tests performed, angular path, depth perception, rotation, and smoothness were found to best fit a log-normal distribution (p > 0.1). Bimanual coordination was found to fit a normal distribution (p ≥ 0.067). However, both normal and log-normal distributions were rejected (p ≤ 0.01) for the metrics of time, volume, and economy of movement. After separating participants into 3 groups based on level of experience with the Fundamentals of Laparoscopic Surgery curriculum, analysis of variance showed significant differences among all group means across the 5 metrics (i.e., angular path, depth perception, rotation, smoothness, and bimanual coordination; p ≤ 0.023).

A proprietary device provided quantitative assessment of laparoscopic skills, which can be used to differentiate among skill levels. Of the 8 tested metrics, 5 fit a normal or log-normal distribution, meaning the scores can statistically be ranked by percentile. Time, volume, and economy of movement did not fit desired distributions. The grading system proved to have construct validity, indicating it may be useful in the longitudinal assessment of laparoscopic skills of surgical trainees.