Some recommendations for developing multidimensional computerized adaptive tests for patient-reported outcomes.

Area of Science:

• Psychometrics
• Health Measurement
• Statistical Modeling

Background:

• Multidimensional item response theory (IRT) and computerized adaptive testing (CAT) are gaining traction in mental health and patient-reported outcome (PRO) measurement.
• While promising, multidimensional CATs present greater application challenges compared to unidimensional approaches.

Purpose of the Study:

• To establish minimal standards for developing multidimensional CATs.
• To provide guidance on the design, evaluation, calibration, efficiency, and precision of multidimensional item banks.

Main Methods:

• Reflection on existing psychometric guidance and discussions from various communities, including PROMIS project guidelines for unidimensional CATs.
• Focus on two key areas: (1) multidimensional item bank development and evaluation, and (2) assessing the efficiency and precision of these banks.
• Suggestion of solutions for simulation studies using real data to avoid overly optimistic efficiency evaluations.

Main Results:

• Item bank development involves distinct construction and psychometric phases.
• Item banks must be sufficiently large to ensure adequate precision across the full range of latent constructs.
• CAT performance evaluation should consider the latent constructs and relevant benchmarks.

Conclusions:

• Multidimensional CATs are complex statistical tools requiring robust theoretical frameworks and rigorous methodological scrutiny.
• Researchers should employ a diverse set of evaluation methods for item banks, clearly document their choices, and validate their approach.