Bias and precision in true-score estimation

We discuss two approaches to estimating the true score from classical test theory, each with a corresponding measure of uncertainty due to measurement error: the classical method with the standard error of measurement (SEM) and Kelley's method with the standard error of estimation (SEE). For both approaches, we examined the bias and sampling variability of the true-score, SEM and SEE estimators, as these properties were largely unknown. For each estimator, we derived analytic expressions for the bias and proposed approximations for practical bias assessment by omitting cumbersome terms. We also proposed a standard error estimator for each estimator. Simulations were used to assess the accuracy of the bias approximations and standard error estimators and to examine the impact of bias and sampling variability on both true-score estimation methods. Results indicate that both the bias approximations and standard error estimators are sufficiently accurate. Whereas the classical true-score estimator is unbiased, Kelley's estimator may exhibit substantial bias for extreme true scores. The bias of the estimated SEM and SEE may also be substantial when reliability is high and the reliability estimator is negatively biased. The impact of sampling variability is typically small.