Reliability of cognitive tests used in Alzheimer's disease

Assessment of cognitive status is a key component of monitoring Alzheimer's patients during the course of their illness. The reliability of a cognitive test is a measure of its reproducibility under replicate conditions. In the classical setting, reliability is defined in three ways: the ratio of the variance of the true scores to the variance of the observed scores; the correlation of observed scores on two parallel forms of the test, and the square of the correlation between the observed score and the true score. In the classical case of independence of true scores and measurement errors, the three definitions are equivalent. Estimation of reliability through analysis of variance techniques and construction of confidence intervals is accomplished when the true scores and errors are normally distributed. This paper examines a non-parametric, probabilistic estimate of reliability as the probability that, given a parallel test, the second set of scores has the same ranking as the first set. In the classical case there is a monotonic relationship between this measure and the reliability. This measure is also linked to Kendall's tau. The performance of the probabilistic measure is compared with the traditional measures in a variety of models, including those with bounded scales, and those with skewed distributions. The ideas are extended to the case of the reliability of change scores and to biased estimators of true scores. In this context truncation models and Bayes estimates of true scores are considered.