Australasian College for Emergency Medicine examiner peer review process: Development and implementation

In 2005, the Australasian College for Emergency Medicine set out to refine the selection, training and development of examiners. Part of this included development of an examiner peer review process.

The objectives of the present study were to observe examiners and seek their opinions on optimal oral examiner techniques, qualities and behaviours, and additionally, to describe how this information was utilized to develop an examiner peer review process.

A qualitative stepwise method was utilized. This began with development of a list of examiner behaviours based on our direct observation of examiners. This was supplemented by a literature search to develop an examiner technique checklist. The checklist items were then put to the Court of Examiners to be rated on a scale of 1-10 for their perceived importance. A modified Delphi technique was utilized to further develop this checklist as an examiner peer review form with behavioural descriptors.

An assessment form was developed with similarly themed items grouped together. This form can now be used as the basis for regular feedback to examiners by appointed senior examiners during a peer review process.

The present article describes the development of a list of optimal examiner attributes, followed by implementation of an examiner peer review process. The authors recommend examiner peer review for high-stakes examinations.