Impact of a Structured Faculty Development Program on Reducing Examiner Variability in Undergraduate Medical Theory...

Area of Science:

• Medical Education
• Assessment Science

Background:

• Examiner variability in medical theory assessments can compromise fairness and reliability.
• Standardized assessments are crucial for learner trust and accurate evaluation.

Purpose of the Study:

• To evaluate the impact of a structured faculty development program (FDP) on reducing examiner variability.
• To improve scoring consistency in undergraduate medical theory assessments using short essay questions (SEQs) and multiple-choice questions (MCQs).

Main Methods:

• A quasi-experimental study involving 32 experienced faculty members, divided into trained and untrained groups.
• Examiners scored anonymized answer scripts pre- and post-intervention.
• Interventions included assessment literacy, rubric design, and examiner calibration for the trained group.

Main Results:

• The FDP significantly reduced examiner variability and improved intra-rater consistency and inter-rater reliability.
• Trained examiners showed marked improvements in scoring accuracy for MCQs (48% to 88%).
• Significant group x time interactions confirmed the intervention's effectiveness.

Conclusions:

• A targeted, short-duration FDP effectively reduced examiner variability and enhanced assessment reliability.
• Implementing such programs promotes fairer, more defensible evaluations in medical education.
• This approach is particularly valuable in resource-constrained settings.