A plea for the proper use of criterion-referenced tests in medical assessment

This paper is aimed at assessment teams which are not steeped in the culture of educational measurement, but, rather, are composed of professionals whose jobs primarily require them to work as clinicians, but whose interest in medical education has given them responsibilities for assessment. It reiterates the difference between criterion-referenced tests and norm-referenced tests. It proposes that those who design and use any assessment in medicine should be clear about which of these approaches to testing they are using.

This paper does not present any new results, but synthesises what is already known about norm-referenced and criterion-referenced tests by reviewing some of the literature. It explains how these two test paradigms lead to different approaches to test design, different measures of reliability and different standard errors of measurement. It shows how these factors may lead to differences in the standards set for some assessments.

Many common medical assessments are assumed to be criterion-referenced but tend to follow norm-referenced practices. Assessment designers should examine the characteristics of each type of assessment to determine which approach is more appropriate and should then apply the correct theories and methods.