Obtaining generalizability coefficients for clinical evaluations

This article discusses the determination of reliability coefficients for clinical evaluation forms by use of generalizability theory. This method is contrasted to some of the commonly used measures of interobserver agreement, interobserver reliability, and classical reliability. The value of this method in terms of making recommendations, such as the number of judges and the number of items on a form, in order to improve reliability is illustrated by two examples. Also, the difference between norm-referenced decisions and criterion-referenced decisions in determining reliability with this method is presented.