Education Research: More Than a Checkbox: Implementing a Narrative-Based Child Neurology Resident Assessment System

Frequent, high-quality feedback is essential for competency-based medical education, yet many graduate medical education assessment systems rely on rating scales and infrequent assessments, limiting actionable feedback. Narrative comments, recognized for their catalytic effect, remain underutilized. The objective of this study was to design and implement a narrative-based, competency-mapped assessment system within a child neurology residency program and evaluate the feasibility of this reform.

The University of Colorado Child Neurology Residency Program redesigned its assessment system in 2023, guided by the Good Assessment framework and the Accreditation Council for Graduate Medical Education Child Neurology Milestones. The new system prioritized adaptability to clinical context, narrative specificity, and integration into existing MedHub workflows. Each form required selection of a competency to assess and justification through narrative comments. Faculty development sessions introduced the structure, expectations, and strategies for writing effective narrative comments. Primary outcomes included volume of assessments and narrative comments. Secondary measures included assessor diversity, time to completion, and distribution of competency selection. We conducted a pre-post analysis comparing a 6-month preintervention period (July-December 2023) with a 6-month postintervention period (July-December 2024).

In the first year of implementation, there were 13 residents. A total of 260 assessments were completed (mean 28.9 per resident), generating 816 narrative comments (mean 90.7 per resident). The median time to completion was 10.5 days (IQR 2-24). Forty-nine unique assessors contributed, including physicians (79.5%) and nonphysicians (20.5%). In the pre-post comparison, assessments increased by 129% (55 to 126), unique assessors by 106% (16 to 33), and narrative comments by 512% (49 to 300). Narrative comments addressed all mapped competencies, with Patient Care and Interpersonal Communication Skills most frequently represented.

A narrative-based, adaptable assessment system is feasible to implement at scale within a child neurology residency program and substantially increases the volume of workplace-based assessments. The system diversified assessors' perspectives and produced narrative comments across all competencies without compromising timeliness. Limitations include single-site design and absence of narrative quality assessment. Future work will focus on evaluating the quality of narrative comments and acceptability of this assessment methodology.