Related Experiment Video
Updated: Dec 6, 2025
Emergency Undocking in Robotic Surgery: A Simulation Curriculum
Published on: May 20, 2018
Establishing Validity for a Vaginal Hysterectomy Simulation Model for Surgical Skills Assessment
Chi Chung Grace Chen1, Ernest G Lockrow, Christopher C DeStephano
1Department of Gynecology and Obstetrics, Johns Hopkins University School of Medicine, Baltimore, Maryland; the Department of Obstetrics and Gynecology, Uniformed Services University of the Health Sciences, Bethesda, Maryland; the Department of Gynecology, Mayo Clinic, Jacksonville, Florida; the Department of Obstetrics and Gynecology, University of California Riverside, Riverside, California; the Department of Obstetrics and Gynecology, Oklahoma University College of Medicine, Oklahoma City, Oklahoma; Augmented Reality Systems Inc., Windham, New Hampshire; and the Department of Obstetrics and Gynecology, Wake Forest School of Medicine, Winston-Salem, North Carolina. At the time of study, all participating institutions were members of the American College of Obstetricians and Gynecologists Simulations Working Group.
Objective:
To use the Messick validity framework for a simulation-based assessment of vaginal hysterectomy skills.
Methods:
Video recordings of physicians at different levels of training and experience performing vaginal hysterectomy on a high-fidelity vaginal surgery model were objectively assessed using a modified 10-item Vaginal Surgical Skills Index, a one-item global scale of overall performance, and a pass-fail criterion. Participants included obstetrics and gynecology trainees and faculty from five institutions. Video recordings were independently assessed by expert surgeons blinded to the identities of the study participants.
Results:
Fifty surgeons (11 faculty, 39 trainees) were assessed. Experience level correlated strongly with both the modified Vaginal Surgical Skills Index and global scale score, with more experienced participants receiving higher scores (Pearson r=0.81, P<.001; Pearson r=0.74, P<.001). Likewise, surgical experience was also moderately correlated with the modified Vaginal Surgical Skills Index and global scale score (Pearson r=0.55, P<.001; Pearson r=0.58, P<.001). The internal consistency of the modified Vaginal Surgical Skills Index was excellent (Cronbach's alpha=0.97). Interrater reliability of the modified Vaginal Surgical Skills Index and global scale score, as measured by the intraclass correlation coefficient, was moderate to good (0.49-0.95; 0.50-0.87). Using the receiver operating characteristic curve and the pass-fail criterion, a modified Vaginal Surgical Skills Index cutoff score of 27 was found to most accurately (area under the curve 0.951, 95% CI 0.917-0.983) differentiate competent from noncompetent surgeons.
Conclusion:
We demonstrated validity evidence for using a high-fidelity vaginal surgery model with the modified Vaginal Surgical Skills Index or global scale score to assess vaginal hysterectomy skills.
