When Do Unifactorial Items Increase the Reliability?

In the context of classical test theory, we study under which conditions addition of an item to a unifactorial test increases the reliability of the test score. We distinguish item weights used to construct a sum score and item factor loadings. We show that the reliability of a weighted sum score increases if the scoring weight of the new item is set sufficiently low. We conclude that any negative contribution to the reliability is not caused by inclusion of the new item per se, but by inadequate weighting of the new item. We also show that if the unweighted sum score is being used with items that have equal error variances, inclusion of a new item increases reliability if the factor loading of the new item is greater than about half of the mean loading of the other items. For standardized items, with loadings based on correlations, we derive lower bounds for the loading of a new item based on the mean and variance of the existing loadings. Instead of removing items when their factor loading is below 0.30, we recommend lowering their scoring weight if they reduce reliability.