The validity and reliability of an OSCE in dentistry

The validity and reliability of a newly-developed objective structured clinical examination for 4th year dental students (OSCE(D)) were estimated by a range of quantitative and qualitative methods. The OSCE(D) consisted of 17 stations in conservation, periodontology and prosthetics. A blueprint was used to match the OSCE stations to clinical subject areas and to 7 skill clusters of clinical competence. Repeat measures of skills were undertaken in the OSCE(D) to maximise its validity and reliability. The primary purposes of the OSCE(D) were to assess clinical competence and to provide feedback to students. The results indicate that the OSCE(D) was intrinsically valid and a better predictor of performance in the final examination than either a concurrent 4th-year examination or Advanced-level university entry grades. The OSCE(D) scored relatively highly on internal consistency (Cronbach's alpha = 0.68). Intra-domain and inter-domain correlations were high and inter-examiner reliability was relatively high (eta 2 coefficients ranged from 0.00 to 0.10). There were no significant differences between performances in the morning and afternoon sessions of the examination so reproducibility is assumed to be high. Some improvement in individual stations and in inter-examiner reliability are required. A set of recommendations based on the experience of designing and testing a dental OSCE are provided.