Australasian College for Emergency Medicine examiner peer review process: Development and implementation

Area of Science:

• Medical Education
• Assessment and Evaluation

Background:

• The Australasian College for Emergency Medicine initiated a project in 2005 to improve examiner selection, training, and development.
• A key component of this initiative was the creation of a peer review process for examiners.

Purpose of the Study:

• To observe oral examiners and gather their insights on optimal techniques, qualities, and behaviors.
• To document the process of developing an examiner peer review system based on these findings.

Main Methods:

• A qualitative, stepwise approach was employed, starting with direct observation of examiner behaviors.
• A literature search informed the development of an examiner technique checklist, which was then rated by the Court of Examiners.
• A modified Delphi technique was used to refine the checklist into a peer review form with behavioral descriptors.

Main Results:

• An assessment form was created, organizing items into thematic groups.
• This form serves as the foundation for regular feedback provided by senior examiners during peer reviews.

Conclusions:

• The study details the creation of a list of optimal examiner attributes and the subsequent implementation of a peer review process.
• The authors advocate for the adoption of examiner peer review in high-stakes examinations to ensure quality and consistency.