Related Experiment Video
Updated: May 14, 2026
Setting Up a Stroke Team Algorithm and Conducting Simulation-based Training in the Emergency Department - A Practical Guide
Published on: January 15, 2017
Preparing Emergency Medicine Residents for the Certifying Exam: A Pilot Study of a Longitudinal Simulation-Based
Jennifer Yee1, Christopher E San Miguel1, Krystin N Miller1
1Department of Emergency Medicine the Ohio State University College of Medicine Columbus Ohio USA.
Background:
The American Board of Emergency Medicine (ABEM) has recently introduced a new Certifying Exam, which incorporates assessment of performance in a simulated clinical environment. Simulation-based assessments allow for reproducible, controlled evaluations. The residency program has a responsibility to prepare residents for the Certifying Exam and to verify resident competency. We sought to determine the feasibility of a simulation-based assessment program encompassing three domains (resuscitation, procedural skills, and communication) with assessments against pre-determined Minimum Competency Scores (MCS) to provide evidence for program director attestations of competence.
Methods:
Eight stations were chosen based on ABEM's Certifying Exam content, the Accreditation Council for Graduate Medical Education (ACGME) Milestones, and proposed Entrustable Professional Activities. Simulation-based case scenarios and assessment checklists were created with standard settings performed using the Mastery Angoff process. These stations were categorized into three different domains: resuscitation (Adult Medical, Pediatric Medical, Trauma, and Neonatal), procedural skills (Direct Laryngoscopy, Cricothyrotomy, and Central Venous Catheter Placement), and communication (Breaking Bad News). Results were summarized with descriptive statistics.
Results:
Thirty emergency medicine residents at an academic Midwestern residency program underwent assessments over one year. None of the residents achieved the MCS on all eight assessments. Residents achieved the MCS on an average of 4.24 stations. There was large performance variability, particularly for the Pediatric Medical (27%-100% items correct), Neonatal Resuscitation (30%-100%), and Breaking Bad News (31%-100%) scenarios. A greater percentage of residents met the MCS for procedural assessments.
Conclusions:
This pilot study demonstrated the feasibility of a simulation-based assessment program designed to provide objective evidence of clinical competency. Assessment outcomes and associated feedback can be used by learners to guide the development of educational plans for performance improvement. Educators can use the resulting data to assess readiness for independent practice, prepare for the Certifying Exam, and identify content areas that may require additional program-wide education.
