Development and implementation of the specialty certificate examinations

Area of Science:

• Medical Education
• Physician Training
• Assessment Methods

Background:

• The Federation of Royal Colleges of Physicians, in partnership with specialist societies, initiated knowledge-based assessments for medical specialty training following successful pilots in 2006.
• This initiative involved over 400 physicians, including the recruitment and training of question writers and the establishment of examining boards and standard-setting groups across 11 medical specialties.

Purpose of the Study:

• To analyze the outcomes of the newly developed specialty certificate examinations.
• To evaluate the effectiveness and reliability of these computer-based assessments in medical specialty training.

Main Methods:

• Analysis of 14 examination diets involving 948 candidates, with 72% in UK training posts.
• Computer-based testing administered in UK and overseas centers.
• Calculation of pass rates and reliability coefficients.

Main Results:

• A total of 948 candidates took the examination, with 786 in the UK and 162 overseas.
• Pass rates for UK trainees consistently exceeded 80%.
• Reliability coefficients were well above 0.8, indicating high assessment consistency.

Conclusions:

• The implemented specialty certificate examinations are effective and reliable tools for assessing physicians in specialty training.
• The program has successfully scaled across 11 medical specialties with positive outcomes for UK trainees.