Primary FRSC (Eng). I. Evaluation of assessment techniques

Area of Science:

• Medical Education
• Surgical Training Assessment

Background:

• The Fellowship of the Royal College of Surgeons of England (FRCS) Primary Examination is a critical assessment for surgical trainees.
• Evaluating the reliability and inter-component correlations within this examination is essential for ensuring fair and accurate candidate assessment.

Purpose of the Study:

• To analyze the correlations between scores from multiple choice questions (MCQ), written papers, and oral interviews in the FRCS Primary Examination.
• To assess the reproducibility of written paper scoring.
• To determine the contribution of each assessment component to the overall evaluation.

Main Methods:

• Statistical analysis of candidate scores across five consecutive FRCS Primary Examinations.
• Examination of correlations between MCQ, written paper, and oral interview scores.
• Evaluation of the reproducibility of written paper marking.

Main Results:

• Significant correlations were found between all examined assessment components (MCQ, written, oral).
• The reproducibility of written paper scoring under a close-marking system was found to be high.
• The oral interview score demonstrated the highest contribution to the overall assessment score, while the MCQ paper contributed the least.

Conclusions:

• All components of the FRCS Primary Examination are necessary for a reliable overall assessment.
• The oral interview is the most influential component in determining a candidate's overall performance.
• The written paper marking system is reliable and reproducible, contributing significantly to the examination's validity.