Effect of percentage cut-off scores and scale point variation on preclinical project evaluation

Several modifications were introduced in a criterion-referenced evaluation system to determine which modification would result in optimal evaluator reliability and performance differentiation on two representative wax-up projects. Variables investigated included the use of percentages as cut-off scores and the number of scoring options. The use of percentage cut-off scores to define minimal competency significantly increased intrajudge reliability for one of the two projects. Two-option scoring resulted in significantly higher rater agreement than three-option scoring in half of the comparisons.