Intraclass correlations as estimates of interrater reliability in nursing research

Intraclass correlation coefficients are useful statistics for estimating interrater reliability. The ICC provides a means for quantifying the level of rater agreement as well as rater consistency. The ICC is easier to use than the Pearson r when more than two raters are involved and can be computed when data are missing on some subjects (Haggard, 1958). Use of this statistic allows the researcher to decide whether or not to include rater effects in estimating IRR and to determine the precision of the reliability estimate. Information about the various types of intraclass correlations and their use is frequently absent from psychometric references commonly used by nurse researchers, resulting in confusion about correct usage and interpretation. Because different values are obtained depending on which ICC formula is selected, ICC formulae reported in the literature can have varying interpretations. For this reason, it is important for researchers to become familiar with the various forms of intraclass correlations and to report the version used in their calculations and the rationale for their choice.