Evaluating Computer Vision and Augmented Reality Guidance to Optimize Weapon Training Safety and Performance

Area of Science:

• Military operational medicine
• Human factors engineering
• Computer vision and machine learning

Background:

• Accurate mortar use is critical for mission success and minimizing Warfighter health hazards like blast overpressure exposure.
• Military training involves dynamic environments where human factors like fatigue can increase error risk.
• The Augmented Reality Guidance System (ARGS) was developed to enhance training and monitor exposures.

Purpose of the Study:

• To develop and evaluate the Augmented Reality Guidance System (ARGS) for supporting Department of War (DoW) decision making.
• To enhance military training effectiveness and serve as a force multiplier.
• To support public health initiatives, including monitoring blast exposures.

Main Methods:

• ARGS utilizes machine learning and computer vision with a ZED stereo-camera for real-time feedback.
• The system evaluates protective equipment use, personnel location relative to weapon systems, and weapon system operation.
• Data was collected across three DoW mortar training events, incorporating real-time facial blurring for privacy.

Main Results:

• ARGS models accurately identified helmet use, head position relative to mortar muzzle, and personnel downrange.
• The system achieved 99% accuracy in detecting mortar rounds across training events.
• High instances of personnel positioned unsafely near weapon systems were identified, indicating training improvement opportunities.

Conclusions:

• ARGS demonstrates potential for real-time range safety support and after-action reviews to improve training.
• The system effectively identifies and communicates safety deviations during training.
• ARGS can aid in monitoring and mitigating blast overpressure exposure, ultimately reducing injuries.