Utilizing Augmented Reality to Evaluate Service Member Team Performance

Decision-making is a complex process that relies on situational awareness and experience to create a potential list of actions while weighing the risks and benefits of each action. There is a paucity of data evaluating decision-making for individual service members (SM) during the performance of team-based military-relevant activities. Understanding individual performance and decision-making within the context of a team-based activity has potential to aid in the detection and management of mild traumatic brain injuries and assist with safe and timely return-to-duty decision making. The aim of this project was to evaluate cognitive and motor performance in healthy SM during an augmented reality military specific, team-based activity.

Data from 110 SMs from Fort Moore Georgia were analyzed for this project. Service members completed 3 augmented reality room breaching and clearing scenarios (Empty Room, Civilian/Combatant, and Incorrect Position of a unit member) with 3 avatar team members. Participants wore a Microsoft HoloLens 2 (HL2) device and used a replica M4 weapon (Haptech Defense Systems) during scenarios. Three-dimensional position data from the HL2 headset was used to compute temporal measures of room breaching and clearing events while the number and timing of weapon discharge was monitored by the M4. Temporal outcomes included time to enter room, time to fire first shot, time in fatal funnel, and total trial time while motor outcomes were distance traveled and average movement velocity.

Pairwise comparisons between the Incorrect Position scenario and the Civilian/Combatant scenario demonstrated no difference in time to enter the room (2.36 seconds in both scenarios). Time to fire the first shot in the Civilian/Combatant scenario was longer (0.97 seconds to 0.58 seconds) while time in fatal funnel (2.58 seconds to 3.31 seconds) and time to trial completion (7.46 seconds to 8.41 seconds) were significantly longer in the Incorrect Position scenario.

Reaction time to fire the first shot, time in the fatal funnel, and total trial time reflect a change in information-processing and decision-making capabilities during military specific, ecological, team-based scenarios when altering the environment inside of the room and modifying avatar movements. Future studies are planned to evaluate the effects of mild traumatic brain injury on specific aspects of military team performance.