Does response shift impact interpretation of change even among scales developed using item response theory?

Response-shift effects impact the interpretation of change in quality-of-life (QOL) measures developed with classical test theory (CTT) methods. This study evaluated the impact of response shift on measures developed using Item Response Theory (IRT), as compared to CTT.

Chronically ill patients and caregivers (n = 1481) participated in a web-based survey at baseline and 17 months later. Patients completed the IRT-based PROMIS-10; NeuroQOL Applied Cognition, Positive Affect & Well-Being short-forms; and the CTT-based Ryff Environmental Mastery subscale. Response-shift effects were evaluated using regression residual modeling and the QOL Appraisal Profile-v2. The sample was divided into positive and negative catalyst groups on the basis of marital, work, job-status, and comorbidity change. Regression models predicted residualized QOL change scores as a function of catalysts and appraisal changes.

In this sample 859 (58%) reported a catalyst. No catalyst was associated with change in scales developed using IRT, but positive work change was associated with the CTT-based measure. Catalyst variables were associated with changes in appraisal, which in turn were related to all outcomes, particularly for global mental health after a positive work-change.

Appraisal processes are relevant to interpreting IRT measures, particularly for global mental health in the face of life changes.