Related Experiment Videos
The impact of faculty development interventions on supervisor narrative feedback quality in competency-based medical
Rob Woods1, Max Spadafore2, Brent Thoma3
1Department of Emergency Medicine, University of Saskatchewan, Saskatoon, SK, Canada. Rob.woods@usask.ca.
Objectives:
Improving the feedback that supervisors provide trainees in medical education is challenging. The development of automated methods of rating the quality of narrative feedback has created an opportunity to quantify the impact of interventions focused on this goal.
Methods:
We used a previously developed natural language processing model based on the Quality Assessment of Learning (QuAL) score to rate the quality of each comment in our Royal College Emergency Medicine specialty training program from July 2018 to June 2024. We quantified the effectiveness of three faculty development interventions (e-mail feedback, audit and feedback workshops, and assessment form modifications) by comparing pre- and post-intervention QuAL scores.
Results:
During the study period 9709 narrative comments were scored. The average QuAL score rose from 3.2/5 in 2019 to 4.5/5 in 2024. All interventions resulted in an overall improvement in the mean QuAL score as measured by assessor weighted mean: individual e-mail with suggestions for improvement (35.1%), assessment form template change (24.3%), workshop (19.1%) and individual e-mail thanking them for their high-quality comments (11.0%).
Conclusions:
By combining a multi-faceted faculty development approach with a scalable measurement of our outcome, we have demonstrated an improvement in supervisor narrative comment quality and quantified the impact of each intervention. While it is likely that the interventions were complementary, individual e-mails to faculty with suggestions for improvement and modifications to the assessment form were associated with largest improvements in quality. These results may guide to other programs seeking to improve the quality of supervisor narrative comments.
Related Concept Videos
Types of Reports I: Hand-off Report
Following are the key components and categories of hand-off reports:
Purpose and Process:
Nursing Process for Patient and Caregiver Teaching III: Evaluation and Documentation
Nurses can use several methods to evaluate patient outcomes. For example, oral questions can assess cognitive learning, patient...
Nursing Evaluation
Section...
Methods of Documentation V: CBE
In CBE, healthcare professionals establish predefined standards of practice that define what constitutes...
Self-Evaluation: Self-Enhancement and Self-Verification
Introduction to Documentation and Reporting
Nursing documentation records essential information and details regarding a patient's care and treatment in written or electronic form. It is a critical aspect of nursing practice that involves documenting assessments, interventions, outcomes, and other relevant details about a patient's health status.
Documentation maps the patient's health journey by creating a comprehensive and precise...