Development and Initial Evaluation of Specific Immersive Competence in Virtual Reality-Based Medical Assessments:...

Virtual reality (VR) is increasingly used in medical education for training and examination purposes; yet, learners' performance in VR-based assessments may be influenced by more than clinical competence alone. Immersive competence (IC) has been proposed as a relevant factor in VR-based performance assessment. While general IC captures application-independent VR interaction skills, domain-specific applications may require an additional construct, specific immersive competence (specific IC), reflecting context-dependent interaction proficiency.

This study aimed to develop and examine the initial psychometric characteristics of a newly developed in situ checklist for assessing specific IC, to explore the relationship between specific and general IC as well as related human abilities and characteristics, and to capture preliminary associations between both IC measures and medical performance in a VR-based objective structured clinical examination (OSCE) station.

In this observational follow-up study, 21 final-year medical students who had previously completed a curricular OSCE including a VR-based emergency medicine station were recruited. General IC was assessed using the VR competence app, and specific IC using a checklist embedded in the original VR simulation. Additional measures included self-reported technological affinity, spatial ability, and OSCE performance scores. Analyses focused on descriptive statistics and exploratory associations, including item difficulty, item-total correlations, and internal consistency for the specific IC checklist. Pearson r was used to examine associations among variables.

The final 13-item specific IC checklist demonstrated acceptable internal consistency for an early-stage instrument (Cronbach α=0.79) and balanced item difficulty (mean P 0.56, SD 0.28). Specific IC showed a strong exploratory association with general IC (r=0.56; P=.008) and with prior 3D application experience (r=0.57; P=.007), but no relevant association with spatial ability. Both general and specific IC showed borderline, moderate associations with VR-OSCE performance (r=0.41; P=.06 and r=0.37; P=.09, respectively), while neither was related to overall analog OSCE performance.

In this pilot sample, specific IC emerged as a psychometrically accessible construct aimed to capture context-sensitive VR interaction skills. The proposed in situ approach offers a feasible and scalable method to assess this construct within domain-specific VR applications. Although associations with medical performance were exploratory and limited by sample size, the findings suggest the relevance of IC as a potential source of construct-irrelevant variance in VR-based assessments and support further investigation in larger, confirmatory studies.