The Optimizing Whiteboard Lectures (OWL) Checklist: Validation of a Novel "Chalk Talk" Assessment Tool

Internal medicine (IM) trainees routinely use whiteboard mini-lectures (WMLs) to teach in clinical settings. Despite their widespread use, there is no standardized, evidence-based assessment tool to evaluate WML quality and structure formative feedback.

To develop and collect validity evidence for the Optimizing Whiteboard Lectures (OWL) Checklist, a novel WML assessment tool, using Kane's four-stage model DESIGN: Prospective, multi-center observational study.

Fifty-one IM residents, fellows, and CMRs delivered 56 WMLs. Twenty-four raters (attendings, fellows, and chief medical residents (CMRs)) performed a total of 91 (43 individual and 48 paired or multi-rater) WML observations.

The OWL Checklist comprises 20 dichotomous behavioral items and a global measure of teaching performance for a maximum score of 24. We collated validity evidence across Kane's four inferences: (1) scoring, through instrument development, rater training, and rater perception surveys; (2) generalization, via inter-rater reliability; (3) extrapolation, by stratifying scores across post-graduate year (PGY) level and by rater perception surveys; and (4) implications, via teacher perception surveys.

We found strong reliability among raters (κ = 0.86) and a significant upward trend in OWL Checklist scores from PGY-1 to CMRs (p = 0.004). Nineteen raters (79%) completed rater perception surveys and endorsed the OWL Checklist as objective (ɑ = 0.68) and reflective of whiteboard teaching best practices (ɑ = 0.79). Forty-four trainees (86%) completed teacher perception surveys and indicated that the OWL Checklist facilitated useful feedback on their whiteboard teaching skillset (98%) that they planned to incorporate into their next WML (100%).

We collected validity evidence for all four stages of Kane's model. This research addresses an important gap in IM trainee evaluation by validating a novel assessment tool that codifies best practices for whiteboard teaching skills and facilitates formative feedback on WMLs. The OWL Checklist has the potential to be utilized in summative assessment.

Not applicable.