Stopping Rules for Computer Adaptive Testing When Item Banks Have Nonuniform Information.

Area of Science:

• Psychometrics
• Computerized Adaptive Testing (CAT)
• Health Outcomes Measurement

Background:

• Standard error (SE) stopping rules in CATs are effective with comprehensive item banks.
• Patient-reported outcome measures often have item banks targeting specific trait levels, limiting informativeness for some individuals.
• Existing SE rules may administer excessive questions when item banks lack depth.

Purpose of the Study:

• To introduce and evaluate the predicted standard error reduction (PSER) stopping rule for CATs.
• To compare the performance of PSER against the SE stopping rule.
• To demonstrate PSER's ability to balance accuracy and efficiency in CATs.

Main Methods:

• Simulated data from PROMIS Anxiety and Physical Function item banks were used.
• The PSER algorithm's parameters were tuned to optimize CAT performance.
• CAT performance was evaluated based on accuracy and efficiency metrics.

Main Results:

• Tuning PSER parameters significantly impacted CAT performance.
• The PSER stopping rule outperformed the SE stopping rule overall.
• PSER was particularly effective for individuals whose traits were not well-targeted by the item bank.
• PSER maintained similar item administration counts across the trait continuum compared to SE.

Conclusions:

• The PSER stopping rule offers an effective approach to balancing precision and efficiency in CATs.
• PSER is advantageous in situations with limited item bank diversity.
• Optimizing PSER parameters allows for tailored trade-offs between accuracy and efficiency.