Related Experiment Videos
Design, implementation, and evaluation of a programmatic assessment model in an emergency medicine residency: a
İbrahim Ulaş Özturan1,2,3, Dilek Kitapcioglu1, Prattama Santoso Utomo3,4
1Department of Medical Education, Acibadem University, Istanbul, Türkiye.
Background:
Programmatic assessment (PA) is widely endorsed in postgraduate medical education, but empirical accounts of real-world implementation in emergency medicine (EM) remain limited and are concentrated in North American settings. We piloted and evaluated the design, implementation, and outcomes of a PA model in an EM residency at a high-volume tertiary care center in Türkiye.
Methods:
A mixed-methods study was conducted over an eight-month pilot (November 2024-June 2025). Program design and evaluation followed a Theory of Change framework applied across three sequential phases: a structured gap analysis informed by external medical-education experts, competency-based blueprint development across six domains, and pilot implementation in a small core-faculty and resident cohort. Outcomes included feasibility (assessment volume and participation), cohort-level educational impact (longitudinal performance via paired-sample comparisons), and acceptability (as measured by an end-of-pilot satisfaction survey and qualitative feedback).
Results:
A total of 3,107 assessment data points were documented across all six competency domains. Among 16 residents with paired baseline and follow-up data, cohort-level relative performance increased significantly in five of six domains. Evidence-Based Medicine and Lifelong Learning did not change significantly. The survey response rate was 89.5%, and 82.4% of respondents endorsed continuing the program. Acceptability data converged across streams: residents simultaneously valued the multisource perspective and voiced concerns about feedback culture, with the burden of form completion as the dominant practical concern.
Conclusions:
Implementation was feasible, accompanied by positive change in cohort-level relative performance, and was broadly acceptable. The most informative finding was a tension between the system's formative design intent and how some residents experienced it, which directly informs next-cycle priorities focused on feedback culture, technology integration, and sustainability.
