When Cronbach's Alpha Is Not Enough: Implications for the Clinical Use of Vision-Related Patient-Reported Outcomes

Patient-reported outcome measures (PROMs) in vision science are commonly evaluated using Cronbach's alpha, yet Rasch-based statistics may provide different information about their reliability. This study compared Cronbach's alpha, a Classical Test Theory (CTT) index, with Rasch person reliability and the person separation ratio across three vision-related questionnaires: the Symptom Questionnaire for Visual Dysfunctions (SQVD; n = 434), the Ocular Surface Disease Index (OSDI; n = 294), and the Quality of Vision questionnaire (QoV; n = 288), using retrospective data from unselected clinical populations. Cronbach's alpha indicated acceptable internal consistency across all questionnaires (α = 0.80-0.87), whereas Rasch person reliability was lower (0.62-0.80) and person separation ratios ranged from 1.29 to 2.00, indicating more limited discrimination. Item reliability was generally high (0.90-0.97) overall but declined in subgroups with restricted score variability. Discrepancies were most pronounced in the OSDI, where estimates approached their theoretical floor when subgroups were defined by the questionnaire's own scores. Cronbach's alpha and Rasch-based statistics are not interchangeable and capture different aspects of measurement precision: alpha may overestimate internal consistency, whereas person reliability and the separation ratio provide complementary information about discriminatory capacity. These findings support reporting both CTT and Rasch-based reliability statistics when evaluating vision-related PROMs in clinical populations.