Using generalizability theory for the estimation of reliability of a patient classification system

This article discusses the measurement issues associated with estimating the reliability of patient classification systems (PCSs). Generalizability theory is proposed as an approach to overcome the limitations of traditional methods of estimating reliability of PCSs. The results of a demonstration study in which generalizability theory is used to support the reliability of a PCS are reported. A coefficient of generalizability, analogous to the reliability coefficient, was computed based on the variance components estimated. The generalizability coefficient for the total PCS score was .034, which increased to .650 when one item was deleted. The generalizability coefficient for individual items ranged from .053 to .961. Suggestions for further instrument development are offered.