Methods for Usability Testing an Educational Simulation: A Case-Study Using a Maternal Telehealth Scenario.

Area of Science:

• Medical Education
• Health Informatics
• Simulation Technology

Background:

• Access to high-quality prenatal and obstetrics care is limited for many women.
• Telemedicine offers a potential solution to bridge this gap by training healthcare providers.

Purpose of the Study:

• To develop and evaluate a telehealth simulation for training resident physicians in managing gestational diabetes.
• To assess the simulation's usability, fidelity, and faculty's ability to measure learner competencies.

Main Methods:

• A mixed-methods usability protocol was adapted for a telehealth simulation.
• Key metrics included task completion rates, perceived difficulty, and faculty scoring reliability.
• The simulation involved resident physicians managing a standardized patient with gestational diabetes.

Main Results:

• Seven out of ten resident physician telehealth competencies were measurable.
• Three specific usability issues were identified that impacted learner performance.
• These issues also limited the faculty's ability to accurately measure three competencies.

Conclusions:

• The telehealth simulation shows promise for training and assessing physician telehealth competencies.
• Addressing identified usability issues is crucial for improving simulation fidelity and competency measurement.
• Further refinement is needed before piloting the simulation with resident physicians.