Comparisons of measures of interclass correlations: the general case of unequal group size

A problem often encountered in epidemiology is the evaluation of the validity of a short questionnaire for diet or physical activity administered to large numbers of subjects, where the gold standard is a diet record or physical activity diary. It is well known that random measurement error can attenuate the interclass correlation coefficient (validity coefficient) between these two variables. Several authors have proposed a de-attenuated (or corrected) correlation coefficient which is an estimate of the true correlation between the two variables after removing the effect of random measurement error. By true correlation we mean the correlation between the questionnaire and the mean of a large or 'infinite' number of diaries (representing the truth). In this paper the authors propose three methods (two ad hoc methods, the pairwise and weighted sib-mean estimators, and the maximum likelihood with confidence limits computed using the Wald statistic and profile likelihood approaches) to estimate the true correlation between a single questionnaire and the mean of an infinite number of follow-up diaries, in the general case where an unequal number of diaries are available for each individual. A simulation study is done under the assumption that the measured variables are normally distributed. Under the null hypothesis of no correlation between the questionnaires and the diaries, all methods had negligible biases. In cases closer to what is usually seen in practice (true correlation between 0.4 and 0.6), the degree of bias and coverage probability depends heavily on the reliability (intraclass correlation) of the diaries. The maximum likelihood estimator with confidence intervals computed by the profile likelihood approach, while not systematically outperforming the other methods, is shown to be the best of the three proposed approaches.