Related Experiment Video
Updated: Jun 25, 2026
The Immersive Cleveland Clinic Virtual Reality Shopping Platform for the Assessment of Instrumental Activities of Daily Living
Published on: July 28, 2022
Immersive Virtual Reality Simulation-Based Assessments in Orthopedic Surgery: A Systematic Scoping Review
Mads Emil Jacobsen1,2,3, Rasmus Drews Wenda1, Mikkel Engell Sandager Nielsen2,3
1Center for Orthopaedic Research and Innovation (CORI), Department of Orthopaedic Surgery, Central and West Zealand Hospital, Slagelse, Denmark.
Background:
Immersive virtual reality (iVR) is increasingly used in orthopedic surgical training, and many platforms generate automated performance metrics that enable scalable competency-based training programs. However, the extent to which iVR-based assessments are supported by validity evidence remains unclear. The purpose of this study was to conduct a scoping review of iVR-based assessments in orthopedic surgery and map validity evidence using Messick's framework.
Methods:
Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews guidelines, we searched PubMed and Embase for original studies evaluating technical performance within orthopedic iVR simulations. Two reviewers independently screened studies and extracted data. They appraised assessment validity evidence mapped to Messick's framework using the Ghaderi scoring instrument and evaluated methodological quality using the Medical Education Research Study Quality Instrument (MERSQI). Data were synthesized descriptively.
Results:
Eighteen studies spanning 11 orthopedic procedures met the inclusion criteria. Validity evidence supporting score interpretation was limited across all 5 of Messick's domains: Total Ghaderi scores ranged from 0 to 7 (median 2.5) of 15, with 0 being the most frequent domain-level rating. Relations to other variables was the most addressed domain, while internal structure and consequences evidence were largely absent. By contrast, methodological quality was generally high (median MERSQI 13.5 of 18), but reporting quality was inconsistent.
Conclusion:
Despite the growing use of iVR in orthopedic training, current iVR-based assessments lack sufficient validity evidence to support meaningful interpretation of performance scores. The field risks mistaking technological sophistication for assessment quality. Strengthening iVR-based assessment requires clearer construct definitions, systematic validity evidence across Messick's domains, and improved reporting practices.
Level Of Evidence:
Level III. See Instructions for Authors for a complete description of levels of evidence.
