Related Experiment Video
Updated: May 13, 2026
Development of a Virtual Reality Assessment of Everyday Living Skills
Published on: April 24, 2014
A virtual reality test to evaluate dismounted soldiers' cognitive and psychomotor performance in an operationally
Matthijs Koedijk1, Annemarie Landman1, Charelle Bottenheft1
1TNO Human Factors Research Institute, Soesterberg, Netherlands.
Background:
Validated and reliable tests are necessary to evaluate the effectiveness of, for example, (data-driven) selection, education and training, and human enhancement interventions, in high-risk professions like the military. Therefore, the current paper describes the development of a test for measuring dismounted soldiers' ability to make quick decisions in unpredictable close-quarters battle (room-clearing) contexts.
Methods:
A group of 15 Dutch Marines Special Operation Forces (SOF) operators and 17 Dutch Army SOF-support specialists individually performed a test in Virtual Reality (VR). Participants could physically walk, shoot, and verbally communicate with opponents. Fifteen performance items indicative of situational awareness (SA) of threats were scored by subject-matter experts, shoot/do not shoot errors were counted, and visual response times (VRTs) in shooting were obtained. Eight room-clearing scenarios were performed twice: once before (pretest) and once following a night of sleep deprivation (posttest), to measure test-retest reliability and sensitivity to a typical military stressor.
Results:
The SA items demonstrated adequate internal consistency, and there was a significant test-retest correlation. SOF operators had significantly higher SA item scores (p < 0.05) and made fewer shoot/do not shoot errors (p < 0.05) than SOF-support specialists. VRTs showed no significant test-retest correlation or effects. Sleep deprivation had no significant effect on any of the performance measures.
Conclusion:
The developed test methodology offers a means to obtain embedded measures of SA, as well as shoot/do not shoot decisions, but VRT measurement appeared to be unreliable. Performance measures were not sensitive to effects of sleep deprivation, possibly due to a counteracting learning effect and limitations in timing of the post-test. The reliability checks of the SA measures were promising, indicating that this study contributes to advancing methodologies for evaluating human enhancement interventions on performance in operationally relevant settings. It is advised to incorporate team performance measures to enhance realism and integrate digitally obtained metrics to minimize observer bias.
