Empirical Priors in Polytomous Computerized Adaptive Tests: Risks and Rewards in Clinical Settings

Area of Science:

• Psychometrics
• Clinical Assessment
• Health Services Research

Background:

• Empirical prior information improves computerized adaptive tests (CATs) in education.
• Its utility in clinical settings, with smaller item banks and polytomous items, remains less understood.
• Clinical CATs are often very short, raising questions about the impact of prior information.

Purpose of the Study:

• To explore the risks and rewards of incorporating prior participant information into CATs within clinical contexts.
• To simulate the effects of prior precision and bias on CAT efficiency and accuracy.
• To evaluate the practical implications for test length and estimation bias in clinical assessments.

Main Methods:

• Two simulation studies were conducted, using applied clinical examples.
• Simulation 1 independently manipulated prior precision and bias.
• Simulation 2 used more realistic combinations of prior bias and precision.

Main Results:

• A precise personalized prior can significantly increase CAT efficiency.
• Overly precise but biased priors risk longer tests or biased estimates, which can be mitigated by setting minimum item counts or less precise priors.
• In realistic scenarios, empirical priors reduced CAT length by a median of 20% in 68% of cases, compared to standard normal priors.

Conclusions:

• Incorporating empirical prior information into CATs is a feasible method to enhance efficiency in clinical settings.
• This approach can reduce test burden for both patients and clinicians.
• Careful consideration of prior precision and bias is essential to maximize benefits and minimize risks.