Education Research: Quality and Validity Evidence for a National In-Training Examination for Epilepsy Fellows.

Area of Science:

• Medical Education
• Neurology
• Assessment and Evaluation

Background:

• Epilepsy education requires structured formative assessment tools due to recent advancements.
• The American Epilepsy Society developed the Epilepsy Fellowship In-Training Examination (EpiFITE) for fellows.
• The EpiFITE aims to enhance fellow assessment, stimulate program improvement, and guide learning.

Purpose of the Study:

• To explore validity evidence for the Epilepsy Fellowship In-Training Examination (EpiFITE).
• To assess the EpiFITE's effectiveness in formative assessment for epilepsy fellows.
• To determine the examination's utility in guiding future learning and teaching.

Main Methods:

• Content linked to the American Board of Psychiatry and Neurology blueprint.
• Expert item development and internal structure analysis (internal consistency, item analysis).
• Surveys of fellows and directors on examination experience and impact on learning and programs.

Main Results:

• The EpiFITE demonstrated stable mean scores and accurate difficulty predictions from 2020-2022.
• High internal consistency (Cronbach α 0.76-0.81) and acceptable item discrimination.
• Fellows and directors found the EpiFITE appropriate and useful for identifying learning needs and program areas for improvement.

Conclusions:

• Multiple sources confirm the quality and validity of the EpiFITE.
• Best practices for developing and evaluating subspecialty examinations were identified.
• The EpiFITE experience can inform the development of similar examinations in other subspecialties.