Related Experiment Video
Updated: Apr 11, 2026
06:48
Emergency Undocking in Robotic Surgery: A Simulation Curriculum
Published on: May 20, 2018
10.5K
Crowd-Sourced Assessment of Technical Skill: A Valid Method for Discriminating Basic Robotic Surgery Skills.
Lee W White1, Timothy M Kowalewski2, Rodney Lee Dockter2
11 School of Medicine, Stanford University, Palo Alto, California. (At time of data collection and analysis: Department of Bioengineering, University of Washington , Seattle, Washington.).
Journal of Endourology
|June 10, 2015
Summary
Crowd workers can accurately assess robotic surgical skills, correlating highly with expert surgeon evaluations. This offers a rapid, cost-effective method for providing feedback to surgeons.
Area of Science:
- Surgical Education and Training
- Robotic Surgery Assessment
- Human Factors in Medicine
Background:
- Accurate assessment of surgical skill is crucial for patient outcomes but often limited by expert availability.
- The internet enables leveraging large crowds for rapid, cost-effective skill evaluations.
- This study explores the potential of crowd-based assessments as an alternative to traditional expert review.
Purpose of the Study:
- To test the hypothesis that crowd worker assessments of surgical skill correlate highly with expert surgeon assessments.
- To evaluate the feasibility of using online crowds for robotic surgical skill assessment.
- To determine if crowd-based assessments are a rapid and low-cost alternative.
Main Methods:
- Forty-nine surgeons performed two robotic surgical skill tasks in a dry laboratory setting.
- Videos of surgical performance were evaluated by 30 crowd workers and 3 expert surgeons using the modified Global Evaluative Assessment of Robotic Skills (GEARS) tool.
- Cronbach's alpha statistic was used to compare mean scores between crowd and expert evaluations.
Main Results:
- Crowd worker GEARS evaluations showed high agreement with expert surgeon ratings.
- Cronbach's alpha values of 0.84 and 0.92 indicated strong correlation for the two tasks, respectively.
- Crowd-based assessments were significantly faster and less expensive to obtain than expert evaluations.
Conclusions:
- Crowd workers can effectively assess basic robotic surgical skills in dry-laboratory tasks, mirroring expert surgeon evaluations.
- Crowd-based assessment provides a viable, adjunctive method for timely feedback to surgical trainees and practitioners.
- This approach offers a scalable and efficient solution to the challenge of surgical skill assessment.
