Enhancing the Predictive Value of Formative Evaluation in Extended Reality Adoption: Addressing the Experience Gap

Area of Science:

• Implementation Science
• Human-Computer Interaction
• Health Technology Adoption

Background:

• Formative evaluation is key for anticipating health technology barriers.
• Extended reality (XR) technologies present unique implementation challenges due to their immersive and embodied nature.
• Traditional pre-use evaluations may not capture experience-dependent properties of XR.

Purpose of the Study:

• To examine potential misclassification of barriers and facilitators in XR adoption using traditional formative evaluation.
• To propose a refinement to formative evaluation methods for XR technologies.
• To improve the alignment of early-stage assessments with real-world implementation.

Main Methods:

• Literature review drawing on implementation science, grounded cognition, and human-computer interaction.
• Analysis of the interpretive problem in applying screen-based evaluation frameworks to XR.
• Proposal of incorporating experiential familiarization into formative evaluation.

Main Results:

• Perception-based formative evaluation, if applied without direct experience, may misinterpret XR barriers and facilitators.
• Stakeholder judgments on XR can be based on anticipation rather than actual use, especially for XR-naive workforces.
• Experiential familiarization can enhance the accuracy of early-stage XR assessments.

Conclusions:

• Traditional formative evaluation may be insufficient for immersive XR health technologies.
• A pragmatic refinement involving brief experiential familiarization is proposed for XR formative evaluation.
• This refinement aims to improve the reliability of implementation assessments for novel health technologies.