Milestone-compatible neurology resident assessments: A role for observable practice activities

Area of Science:

• Medical Education
• Neurology Training
• Competency-Based Assessment

Background:

• US neurology residency programs must report trainee progression across 29 Milestone competency domains.
• Existing trainee assessment systems require adaptation to meet these reporting requirements.

Purpose of the Study:

• To validate neurology resident assessment content using observable practice activities (OPAs).
• To develop assessment formats easily translatable to the Neurology Milestones.

Main Methods:

• A modified Delphi technique established consensus on the importance of 73 neurology OPAs among educators and trainees.
• Content validity scores (CVS) were calculated, with ≥4.0 indicating sufficient validity.

Main Results:

• The mean CVS for all OPAs was 4.4; 78% met the ≥4.0 threshold.
• Trainees rated OPAs as more important than faculty (mean CVS 4.6 vs. 4.4).
• Two assessment formats were developed, one using OPAs alone, another combining OPAs and Milestones.

Conclusions:

• This study offers content validity evidence for neurology resident assessments and formats aligned with Neurology Milestones.
• Perceptions of OPA importance are largely consistent across experience levels.