How assessment drives learning in neurosurgical higher training

Area of Science:

• Medical Education
• Neurosurgery Training
• Assessment Utility

Background:

• Neurosurgical trainee competence certification is vital for public safety in Australia and New Zealand.
• Current assessment methods lack empirical data on their utility (validity, reliability, educational impact).
• Direct study of national certification is limited by small trainee and examiner numbers.

Purpose of the Study:

• To evaluate the validity, reliability, and educational impact of an essay examination used in neurosurgical training.
• To provide indirect evidence of assessment utility using a larger cohort during formative assessment.

Main Methods:

• A short answer essay examination was administered to 59 neurosurgical trainees.
• Examiners (up to nine per question part) marked the essays, with data analyzed.
• Trainee questionnaires (n=48) and semi-structured interviews with high-achieving trainees (n=8) provided further evaluation.

Main Results:

• The essay examination demonstrated low reliability (generalizability coefficient of 0.56 with 6 questions).
• The examination was found to be invalid as an assessment tool.
• Evidence suggests a potential negative impact, promoting declarative knowledge over practical neurosurgical competence.

Conclusions:

• While not directly assessing the Fellowship examination, the findings raise concerns about assessment instruments.
• The study highlights the need to carefully consider how assessment tools influence neurosurgical trainees' learning strategies.
• Emphasis on theoretical knowledge may detract from essential surgical skill acquisition.