A comparison of intra- and interjudge decision consistencies using analytic and holistic scoring criteria

This study explored the effect of scoring criteria on the decision consistency of judges. Intra- and interjudge consistencies were compared when two different scoring criteria were used by the same judges. Analytic criteria included detailed assessments of a student's performance, while holistic criteria included only a subjective global assessment. Data from 18 judges and 9 histotechnology practical examinations were analyzed. Lack of intrajudge decision consistency was observed when the same practical examinations were assessed using analytic and holistic criteria. Interjudge decision consistency also varied when the different scoring criteria were used. More practical examinations were judged as passing when analytic scoring criteria were used. This particular analysis did not support a definitive conclusion concerning the value of holistic and analytic criteria.