Related Experiment Video
Updated: Apr 12, 2026
Evaluating Flight Performance and Eye Movement Patterns Using Virtual Reality Flight Simulator
Published on: May 19, 2023
Testing basic competency in knee arthroscopy using a virtual reality simulator: exploring validity and reliability
Mads Emil Jacobsen1, Morten Jon Andersen2, Claus Ol Hansen3
1Faculty of Health and Medical Sciences, University of Copenhagen, Matthæusgade 48C, Nr. 418, 1666 Copenhagen V, Denmark.
Background:
Diagnostic knee arthroscopy is a common procedure that orthopaedic residents are expected to learn early in their training. Arthroscopy requires a different skill set from traditional open surgery, and many orthopaedic residents feel less prepared for arthroscopic procedures. Virtual reality simulation training and testing provide an opportunity to ensure basic competency before proceeding to supervised procedures in patients.
Methods:
Twenty-six physicians (thirteen novices and thirteen experienced arthroscopic surgeons) were voluntarily recruited to perform a test consisting of five arthroscopic procedures on a knee arthroscopy simulator. Performance was evaluated by obtaining predefined metrics from the simulator for each procedure, and z-scores, describing suboptimal performance, were calculated from the metrics. The intercase reliability of the simulator metrics was explored by calculating an intraclass correlation coefficient. Finally, a pass-or-fail standard was set with use of the contrasting groups method, and the consequences of the pass-or-fail standard were explored.
Results:
One procedure was excluded from the final test because of a lack of validity. The total Z-scores for the four procedures included in the final test showed an intercase reliability of 0.87 (95% confidence interval, 0.78 to 0.93). The total mean z-score (and standard deviation) was 38.6 ± 27.3 points for the novices and 0.0 ± 9.1 points for the experienced surgeons (p < 0.0005). The pass-or-fail standard was set at a total z-score of 15.5 points, resulting in two of the novices passing the test and a single experienced surgeon failing the test.
Conclusions:
By combining four procedures on a virtual reality arthroscopy simulator, it was possible to create a valid, reliable, and feasible test of basic arthroscopic competency and to establish a credible pass-or-fail standard.
Clinical Relevance:
The simulation-based test and pass-or-fail standard could aid in assessing and ensuring basic competency of future orthopaedic residents before proceeding to supervised procedures in patients.
