A general framework for evaluating the reliability of medical measurement systems

Measurements of physical and social phenomena are often used in the decision-making process despite the error introduced by human and context variation. It would be advantageous to a decision maker to know the sources and magnitudes of such errors in using a measurement system or in choosing among alternative measurement systems. This paper describes an approach to determining the reliability of a measurement procedure called generalizability (G) theory. G theory uses the results of a random-effects analysis of variance to estimate the magnitude of error variability associated with each of several potential sources. These estimates can then be used to calculate a number of indices which typify the amount of precision provided by a measurement system in a particular context of usage. The major concepts of G theory are described and illustrated in the context of evaluating a radiologic total lung capacity measurement system.