The risks of thoroughness: Reliability and validity of global ratings and checklists in an OSCE

To compare checklists against global ratings for student performance on each station in an OSCE without the confounder of the global rating scorer having first filled in the checklist.

Subjects were 96 medical students completing their pre-clinical studies, who took an 8 station clinical OSCE. 39 students were assessed with detailed performance checklists; 57 students went through the same stations but were assessed using only a single global rating per station. A subset of 39 students were assessed by two independent raters.

Inter-rater and inter-station reliability of the global rating was the same as for the checklist. Correlation with a concurrent multiple choice test was similar for both formats.

The global rating was found to be as reliable as more traditional checklist scoring. A discussion of the validity of checklist and global scores suggests that global ratings may be superior.