Customizing CAT Stopping Rules for Clinical Settings: A Simulation Study Using the NIH Toolbox Emotion Battery...

Area of Science:

• Psychometrics
• Clinical Assessment
• Health Outcomes Research

Background:

• Patient-reported outcome measures (PROMs) are vital for clinical decision-making and treatment evaluation.
• However, PROMs can impose a significant burden on patients and may lack the reliability needed for clear clinical interpretation.

Purpose of the Study:

• To evaluate alternative stopping rules for computerized adaptive testing (CAT) in the NIH Toolbox® Emotion Battery.
• To assess the impact of these rules on test burden, score precision, and reliability.

Main Methods:

• Computerized adaptive testing (CAT) simulations were performed on adult item banks within the Negative Affect subdomain of the NIH Toolbox®.
• Simulations compared original stopping rules (standard error < 0.3) with modified rules, including SE-change rules, fixed-length CATs, and reduced maximum item counts.
• Measures included administered items (burden), reliability cutoffs (precision), and root mean squared error (score recovery).

Main Results:

• Alternative stopping rules generally reduced burden and increased the proportion of assessments achieving high reliability compared to original rules.
• However, some alternative rules, like the SE-change rule and short fixed-length CATs, increased assessments with low reliability (< 0.85).
• A reduced maximum stopping rule demonstrated the best balance between precision and parsimony.

Conclusions:

• Reducing test burden while maintaining score precision for clinical use presents a significant challenge.
• The choice of stopping rules for PROMs should be tailored to the specific study population and research objectives.