Hybrid Usability Analysis to Improve an Educational Telemedicine Simulation: Task Analysis and Survey Results

Area of Science:

• Medical Education
• Simulation Technology
• Health Professions Education

Background:

• Standardized patients are crucial for medical simulations.
• Ensuring consistent learner experiences and faculty assessment in simulations is challenging.
• Usability testing methods can be adapted for simulation evaluation.

Purpose of the Study:

• To design and evaluate a mixed-methods instrument for assessing medical education simulations.
• To identify usability issues affecting simulation performance and competency assessment.
• To provide an adaptable and reproducible evaluation method for educators.

Main Methods:

• Developed a mixed-methods evaluation instrument based on usability testing principles.
• Collected quantitative data on task completion, competency assessment, and user perceptions.
• Gathered qualitative data on simulation usability issues.

Main Results:

• Half of testers failed to complete telehealth safety checks; one missed an audio/visual cross-check.
• Simulation issues impacted the assessment of clinical data collection, equipment use, and professional standards.
• Qualitative feedback identified actionable improvements for future iterations.

Conclusions:

• The developed instrument effectively identified critical usability flaws in medical simulations.
• Qualitative feedback facilitated targeted improvements, enhancing simulation reliability.
• This reproducible evaluation method can benefit various medical education simulation contexts.