Assessing Competence in Emergency Radiology Using an Online Simulator

Traditional assessments in radiology residency focus on the Medical Expert CanMEDS role and typically rely upon a single or limited static images. We designed an Emergency Radiology Simulator that aimed to assess the breadth of competencies required across Medical and NonMedical Expert domains.

An online simulator with typical emergency cases was administered in October 2015 to Post Graduate Year (PGY) 2-5 residents in Radiology. Residents provided preliminary reports, which were graded for style and content. The simulation also included prioritization, protocoling, counseling, and handover exercises geared to assess NonMedical Expert roles.

Fourty eight residents participated in the simulation. Level of resident was 11 PGY-2, 17 PGY-3, 13 PGY-4, and 7 PGY-5. There was a significant difference in resident performance between PGY-2 residents and those more senior in terms of the Medical Expert role (findings, diagnosis, recommendations, and clinical relevance of reports). Differences in performance between PGY levels were not seen in the NonMedical Expert roles (prioritization, protocoling, counseling, and handover).

Simulation provides an opportunity to assess radiology resident performance across multiple domains. PGY-2 residents performed worse on the Medical Expert domains, although performance did not significantly vary between the other years. This may suggest that competence in Emergency Radiology is achieved early in residency, possibly related to the importance placed on developing skills related to on-call performance during the PGY-2 year. The simulator should be extended to other areas of Radiology, in order to assess the ability to discriminate performance in other subspecialties.