Related Experiment Video
Updated: Jul 5, 2025
Using Learning Outcome Measures to assess Doctoral Nursing Education
Published on: June 21, 2010
The Quality of Assessment for Learning score for evaluating written feedback in anesthesiology postgraduate medical
Eugene K Choo1, Rob Woods2, Mary Ellen Walker1
1Department of Anesthesiology, College of Medicine, University of Saskatchewan, Saskatchewan, Canada.
Background:
Competency based residency programs depend on high quality feedback from the assessment of entrustable professional activities (EPA). The Quality of Assessment for Learning (QuAL) score is a tool developed to rate the quality of narrative comments in workplace-based assessments; it has validity evidence for scoring the quality of narrative feedback provided to emergency medicine residents, but it is unknown whether the QuAL score is reliable in the assessment of narrative feedback in other postgraduate programs.
Methods:
Fifty sets of EPA narratives from a single academic year at our competency based medical education post-graduate anesthesia program were selected by stratified sampling within defined parameters [e.g. resident gender and stage of training, assessor gender, Competency By Design training level, and word count (≥17 or <17 words)]. Two competency committee members and two medical students rated the quality of narrative feedback using a utility score and QuAL score. We used Kendall's tau-b co-efficient to compare the perceived utility of the written feedback to the quality assessed with the QuAL score. The authors used generalizability and decision studies to estimate the reliability and generalizability coefficients.
Results:
Both the faculty's utility scores and QuAL scores (r = 0.646, p < 0.001) and the trainees' utility scores and QuAL scores (r = 0.667, p < 0.001) were moderately correlated. Results from the generalizability studies showed that utility scores were reliable with two raters for both faculty (Epsilon=0.87, Phi=0.86) and trainees (Epsilon=0.88, Phi=0.88).
Conclusions:
The QuAL score is correlated with faculty- and trainee-rated utility of anesthesia EPA feedback. Both faculty and trainees can reliability apply the QuAL score to anesthesia EPA narrative feedback. This tool has the potential to be used for faculty development and program evaluation in Competency Based Medical Education. Other programs could consider replicating our study in their specialty.
Related Concept Videos
01:20Nursing Process for Patient and Caregiver Teaching III: Evaluation and Documentation
Nurses can use several methods to evaluate patient outcomes. For example, oral questions can assess cognitive learning,...
01:15Nursing Evaluation
01:11Guidelines for Writing Outcome
Patient outcomes reflect the patient's response to the goal rather than what the nurse aims to achieve. Terminology should be observable and measurable to avoid the reader's interpretation. The desired outcome should be realistic and achievable in the designated care timeframe. Expected outcomes should align with adjunctive therapies. The outcome should enhance care...
01:08Role of Communication in the Nursing Process III: Evaluation and Documentation
01:25Assessment of apical radial pulse
The A-R pulse assessment involves simultaneous evaluation of the apical and radial pulses. When the apical and radial pulse rates vary, this assessment helps identify a pulse deficit.
Pre-Procedural Preparation
01:23Assessment of Respiration
Subjective Assessment: Nurses interview the patient to gather information directly during the subjective assessment. It includes questions about the individual's medical history, medications, and symptoms, focusing on past respiratory conditions like...