Mixed Reality and Desktop Hand Hygiene Training With Deep Learning-Based Step Recognition and Real-Time Decision...

Area of Science:

• Healthcare technology
• Artificial intelligence in medicine
• Infection control

Background:

• Conventional hand hygiene (HH) monitoring lacks detail on procedural quality and feedback.
• Need for advanced systems to improve HH compliance and prevent healthcare-associated infections.

Purpose of the Study:

• To develop and evaluate a real-time HH training and assessment framework using deep learning and AI.
• To compare the effectiveness of Desktop versus Mixed Reality (MR) interfaces and Concurrent-Feedback Coaching (CFC) versus Uncued Retention Assessment (URA) modes.

Main Methods:

• Developed a framework combining deep learning (YOLOv12-n+MV) for WHO HH step recognition with a decision support engine.
• Deployed the system on Desktop LED and MR (HoloLens 2) platforms, supporting CFC and URA modes.
• Conducted a mixed-methods study with 20 participants evaluating performance, usability, and workload.

Main Results:

• The compact YOLOv12-n+MV model achieved a favorable accuracy-efficiency trade-off for real-time HH recognition.
• Desktop interfaces demonstrated significantly faster, more stable, and less error-prone HH performance compared to MR.
• CFC mode reduced completion time, while URA mode reduced errors, indicating a speed-accuracy trade-off. Desktop also showed higher usability and lower workload.

Conclusions:

• Desktop displays are more suitable for HH training prioritizing efficiency, stability, and lower workload.
• CFC and URA modes can be selectively applied in Desktop or MR workflows to enhance guided learning or independent recall.
• The developed framework offers a novel approach to real-time, quality-focused HH training and assessment in healthcare settings.