Evaluating the Usefulness of Translation Technologies for Emergency Response Communication: A Scenario-Based Study

Area of Science:

• Emergency Medical Services (EMS)
• Health Informatics
• Communication Technology

Background:

• Language barriers significantly impede emergency response and care for Limited English Proficient (LEP) individuals in the U.S.
• Emergency Medical Services (EMS) personnel are exploring automated translation tools to bridge communication gaps with LEP populations.
• The efficacy of these tools in real-world emergency settings remains largely unexamined.

Purpose of the Study:

• To evaluate the potential of two automated translation technologies in enhancing emergency communication.
• To assess the usability and effectiveness of these tools through scenario-based simulations with EMS providers and LEP participants.

Main Methods:

• Developed mock emergency scenarios for simulation sessions involving EMS personnel and Spanish/Mandarin-speaking LEP participants.
• Utilized two translation tools: QuickSpeak (EMS domain-specific) and Google Translate (statistical machine translation).
• Collected feedback via post-session questionnaires and the System Usability Scale (SUS) for EMS providers.

Main Results:

• Overall communication effectiveness remained limited despite tool usage.
• QuickSpeak demonstrated higher usability scores (SUS: 65.3) compared to Google Translate (SUS: 48.4).
• LEP participants preferred QuickSpeak for understanding questions (92% Chinese, 86% Spanish) over Google Translate (25% Chinese, 29% Spanish), despite appreciating the flexibility of the latter.

Conclusions:

• Emerging translation technologies offer potential for emergency response but require substantial improvements in accuracy and usability.
• Current automated tools are not yet sufficiently reliable for safe deployment in field emergency situations.
• Further development is crucial to ensure effective and safe communication between EMS and LEP individuals.