What is being assessed in the MRCGP oral examination? A qualitative study

Oral examinations are a popular method of assessment within medicine, being capable of measuring candidates' ability to carry out tasks or develop skills (operational knowledge). One example of this is the oral examination for membership of the Royal College of General Practitioners (RCGP), which is designed to assess candidates' decision-making skills and the professional values that underpin these decisions. While the reliability of oral examinations has been investigated, to date, little is known about their ability to measure what they set out to measure (validity).

To investigate the content validity of the MRCGP oral examination, with particular focus on its ability to assess the process of decision-making.

An evaluation of oral examination video recordings, using qualitative methods.

The MRCGP oral examinations are video recorded as part of an ongoing quality assurance programme. Fifty of the recordings carried out in 2002 were selected randomly and analysed for content and dialogue patterns reflecting the assessment of the decisionmaking process.

All examiners used the specified contexts outlined in the examination objectives to present candidates with dilemmas. The assessment of decision-making skills, however, was limited by a tendency among examiners to present the candidate with new, more complex dilemmas rather than giving them the opportunity to discuss the implications, make choices and ultimately, justify their decision. Moreover, while examiners frequently asked candidates questions relating to professional values, they rarely asked them to demonstrate how those values support their decisions.

In order that the benefits of oral examination can be fully realised, questions need to be structured in a way that encourages candidates to discuss all stages of the decision-making process.