Impact of Narrative-Focused Feedback Forms in a Pulmonary and Critical Care Medicine Fellowship

Existing formal feedback systems in graduate medical education often rely heavily on intangible Likert Scales linked to ACGME milestones as an end-of-rotation summative assessment. However, more frequent, actionable, formative feedback based on observations is essential to personal growth.

To evaluate the impact of changing from a milestone-based evaluation form to a brief narrative-based feedback form in a Pulmonary Disease and Critical Care Medicine (PCCM) fellowship as an exploratory pilot project. Our aim was to describe change in perception of feedback quality and the impact on Clinical Competency Committee (CCC) decisions.

We compared the quality of written rotation feedback before and after implementing a shorter feedback form on January 1, 2024 for 15 PCCM fellows at a single large academic institution using the Quality of Assessment for Learning (QuAL) score, (July 2023-December 2023 compared to January 2024-July 2024), fellow pre- and post-surveys (December 2023 and June 2024) and a CCC focus group (June 2024).

Overall QuAL score improved significantly after narrative-based feedback form implementation (2.71 pre [n = 75] vs 3.70 post [n = 85], p < 0.001). Surveys had 13/14 (92.8%) fellows respond pre-intervention and 8/14 (57.1%) post-intervention. There was significant improvement in the number of fellows indicating they could identify new learning goals based on written feedback after implementation of the new form (23.1% pre vs 75.0% post, p = 0.03). The CCC focus group with 5 of 8 eligible faculty participating described milestone decisions were better supported by more detailed narratives.

Implementing a brief narrative-based form improved feedback quality and the number of trainees able to create learning goals. The CCC identified that the new form provided better details to support milestone recommendations.