Reconciling information and reliability in scaling local measurement precision: A comment on O'Connor (2017)

O'Connor (2017) illustrates the importance of accurately characterizing how measurement precision impacts effect size inference. As he demonstrates, attenuation of correlations can be much greater than traditional predictions based on classical test theory, due to fluctuating measurement error and other considerations. In his paper, however, O'Connor attempts to rescale information in terms of a local or conditional reliability. Although this is intuitively appealing and useful in certain respects, it can also be misleading and mischaracterize how test information affects reliability, and by extension, effect size inference. Information is a component of reliability, but other factors affect reliability as well, such as variation in the attribute or trait being measured. Translating the findings into practice requires care in quantifying different aspects of measurement precision. (PsycINFO Database Record