Evaluating Nontechnical Skills in US Emergency Departments Using Simulation: Validating and Contextualizing a UK...

Area of Science:

• Medical Education
• Emergency Medicine
• Healthcare Professional Skills Assessment

Background:

• Nontechnical skills (NTS) are crucial for safe and efficient medical care, complementing technical expertise.
• Existing NTS assessment tools may not fully capture the nuances of diverse clinical environments.
• Validating and adapting international assessment tools for specific national contexts is essential.

Purpose of the Study:

• To evaluate the validity and reliability of a 12-element United Kingdom emergency medicine (EM) NTS assessment tool within the United States (US) EM context.
• To identify specific behavioral elements unique to US clinical practice that should be incorporated into NTS assessment.
• To assess the tool's suitability for formative assessment of US EM physicians (EPs).

Main Methods:

• A mixed-methods study employing Kane's validity framework across two phases.
• Phase I: Focus group to assess the appropriateness of the UK NTS tool elements in simulated US EM scenarios.
• Phase II: 208 EPs evaluated video simulations of clinical encounters, assessing NTS reliability using Cronbach's alpha.

Main Results:

• The NTS assessment tool demonstrated acceptable internal consistency (Cronbach's α > 0.7) for most categories, with an exception in teamwork and cooperation.
• Participants proposed four novel behavioral elements relevant to US EM practice.
• Exemplar behaviors for all 12 original elements were suggested, tailored to the US clinical context.

Conclusions:

• The validated UK NTS tool is suitable for formative assessment of US EPs.
• Incorporating US-specific behavioral elements and refining existing ones can optimize the tool's usability and relevance.
• This adapted tool supports the development of essential nontechnical skills in US emergency medicine.