Reducing patient burden of PROMs in healthcare through advanced computerized adaptive testing stopping rules

Application of computerized adaptive testing (CAT) can improve the assessment of patient-reported health outcomes by reducing patient burden. We aimed to reduce patient burden of CATs further by optimizing a standard error reduction stopping rule (SER; minimum change in SE(θ) after each CAT step).

We extracted PROMIS Anxiety and Depressive Symptoms CAT responses (mean age = 13.7, male = 50.3%) from the Dutch-Flemish PROMIS Assessment Center and estimated theta levels (θ) and standard errors (SE(θ)) for each step. The default stopping rules were a minimum/maximum of 4/12 items administered, respectively, or a minimum precision of SE(θ) < 0.32. We imposed increasing SER thresholds (0.01-0.20) and compared the following outcome criteria: mean efficiency of the CAT (M_efficiency; 1 - SE(θ)²/n_items), mean number of items administered (Mn_items), the mean SE(θ) of all respondents (M_SE(θ)), and mean T-score difference compared to default stopping rules (M_∆T).

Default stopping rules showed a mean efficiency of 0.88 and1.27, Mn_items = 9.98 and8.13, and M_SE(θ) = 0.36 and0.38 for respectively the Anxiety and Depressive Symptoms item banks. We optimized the SER value with a differential efficiency function, resulting in shorter, more efficient CATs (Anxiety: mean efficiency = 1.08, Mn_items = 5.58, M_SE(θ) = 4.24, M_∆T = 0.04; Depressive Symptoms: mean efficiency = 1.45, Mn_items = 4.79, M_SE(θ) = 4.15, M_∆T = 0.58). For participants reporting no problems, this results in fewer items administered, but a decrease in measurement accuracy and biased T-scores, which may be relevant depending on the goal of assessment.

We conclude that the current approach allows us to determine an optimal SER threshold that improves measurement efficiency, especially when floor/ceiling effects are present in the target population. The threshold values will vary depending on the θ distribution of the target population and the IRT model parameters.