Feedback in an Entrustment-Based Objective Structured Clinical Examination: Analysis of Content and Scoring Methods

Area of Science:

• Medical Education
• Clinical Assessment
• Healthcare Professional Development

Background:

• Integrating Entrustable Professional Activities (EPAs) into Objective Structured Clinical Examinations (OSCEs) offers timely feedback for residents.
• Concerns regarding the quality of feedback in these assessments persist.
• This study addresses the need to evaluate feedback effectiveness in an entrustment-based OSCE context.

Purpose of the Study:

• To assess the quality and content alignment of verbal feedback from examiners during an entrustment-based OSCE.
• To compare the impact of different scoring methods on feedback quality and alignment with learning objectives.

Main Methods:

• A progress test OSCE for internal medicine residents assessed 7 EPAs.
• Examiner feedback (2-minute duration) was recorded and analyzed using the Quality of Assessment of Learning (QuAL) score.
• Feedback alignment with EPA learning objectives (competency milestones, task-specific abilities) was evaluated.
• A randomized crossover experiment compared 3-point entrustability scales versus task-specific checklists for scoring.

Main Results:

• Twenty-one examiners provided feedback to 67 residents.
• Feedback quality was high (mean QuAL score 4.3/5) and aligned with EPA learning objectives.
• Examiners addressed an average of 2.5 milestones (61%) and 1.2 task-specific abilities (46%).
• Scoring methods showed no significant impact on QuAL scores or alignment with milestones/abilities (P > .13).

Conclusions:

• Examiners provided valuable, high-quality feedback aligned with learning outcomes in the entrustment-based OSCE.
• Feedback quality and alignment were explored as distinct dimensions.
• The scoring methods evaluated did not significantly influence feedback quality or its alignment with learning objectives.