An investigation into the validity and reliability of the marking of students' orthodontic practical work

This study has shown all three examiners to be different; surely a reflection of human nature. One examiner appeared to be strongly influenced by the students' past performance in the course, and the results confirm the findings of Natkin and Guild (1967) that examiners do not always observe the same errors and even when they do, may attach differing importance to the observed deficiency. It was found that an examiner's standard may vary from day to day but, in general, the intra- and inter-examiner correlations were similar to those reported in other studies of undergraduate practical work (Lilley et al., 1968; Silvestri et al., 1979; Goepferd and Kerber, 1980). It may be concluded that: 1. All marking should be done "blind' whenever possible. 2. Marking, once started, should be finished without a break as an examiner's own "pass' standard may vary unduly from one occasion to another. 3. The entire batch of material should be marked by the same examiner, as another could have a significantly different pass standard even though they might agree on the candidates' final order. 4. Because one examiner may subconsciously attach rather more importance to an aspect of the work that another does not notice, there would be a case for two or more examiners carrying out the marking. Their average marks could then be used. In this case, each examiner would, of course, mark the entire batch.