Coefficients of Repeatability for Likely Change: Comparison between PROMIS Computer Adaptive Tests and Short Forms

The study identified necessary score changes to categorize patients as likely to have changed in PROMIS computer adaptive testing (CAT) and to compare these with selected 4-item or 8-item short forms (SF 4a or 8a).

In a large health system symptom management intervention project, 5823 completed PROMIS CAT with the stopping rule of standard error < 3 for anxiety (v1.0), depression (v1.0), pain interference (v1.1), and physical function (v2.0) at baseline and at 4 months. Coefficients of repeatability (CRs) for likely change were determined using 68 % confidence level based on the reliable change index statistics using item response theory standard errors, which were compared with those using 90 % confidence level. CRs for 8-item or 4-item short forms (SFs) were estimated for comparison using established item parameters.

CRs at 68 % confidence ranged from |3| to |4| for CAT and SF 4a and |2| to |3| for SF 8a across domains for baseline scores in the mildly-to-severely impaired ranges. For baseline scores within normal limits across improvement or worsening, CRs at 68 % confidence were |3|-|8| for CAT, |3|-|14| for SF 4a, and |2|-|14| for SF 8a. Across score ranges, directions of change, and domains, relaxing confidence level from 90 % to 68 % lowered the change thresholds by up to 7 points for CAT, 14 points for SF 8a, and 14 points for SF 4a, with the largest reductions for baseline scores within the normal range.

In symptomatic score ranges, smaller and somewhat constant CRs were observed compared to scores within normal ranges. SF 8a achieved CR of 1 point lower than CAT with 4 more items on average in the symptomatic score range. CAT achieved lower CR than either SF length in the healthiest score range.