The Optimal Item Pool Design in Multistage Computerized Adaptive Tests With the p-Optimality Method

Area of Science:

• Psychometrics
• Educational Measurement
• Computerized Adaptive Testing

Background:

• Multistage computerized adaptive tests (MST) require carefully designed item pools.
• Optimizing item pools is crucial for accurate and efficient testing.
• Existing methods may not fully address MST-specific challenges.

Purpose of the Study:

• To extend the p-optimality method for developing optimal item pools in MST.
• To evaluate the performance of p-optimality generated pools under various MST configurations.
• To investigate the impact of exposure control on item pool characteristics.

Main Methods:

• Utilized the p-optimality method within the Rasch model framework.
• Generated simulated optimal item pools with and without exposure control.
• Evaluated 72 simulated item pools using statistical measures on overall and conditional samples.

Main Results:

• P-optimality generated item pools demonstrated sufficient measurement accuracy for all simulated MST designs.
• Exposure control influenced item pool size but not item distribution or characteristics.
• The p-optimality method proved adaptable to MST item pool development.

Conclusions:

• The p-optimality method is a viable approach for constructing MST item pools.
• This method enhances the MST assembly process and improves scoring accuracy.
• The findings support the use of p-optimality for robust adaptive testing item pool design.