Developing Multistage Tests Using D-Scoring Method

Area of Science:

• Educational Measurement
• Psychometrics
• Psychological Measurement

Background:

• The D-scoring method integrates item-level difficulty from Item Response Theory (IRT) with the simplicity of Classical Test Theory (CTT).
• It offers advantages like item difficulty information and score computation reflecting item difficulties, while maintaining CTT's ease of use and relaxed sample size requirements.
• Its unique blend of features has led to rapid adoption in educational and psychological measurement.

Purpose of the Study:

• To develop and evaluate Multistage Testing (MST) mechanisms utilizing the D-scoring method.
• To propose and implement a novel framework for conducting MST simulations under the D-scoring approach.
• To assess the score recovery performance and comparability of D-scoring within MST designs.

Main Methods:

• Development and comparison of multiple MST algorithms tailored for the D-scoring method.
• Implementation of a new simulation framework specifically designed for D-scoring based MST.
• Evaluation of score comparability across different MST routes and measurement precision/efficiency.

Main Results:

• Promising score recovery performance was observed for MST employing the D-scoring method.
• Score comparability was successfully maintained across various MST paths.
• MST utilizing the D-scoring method demonstrated enhanced measurement precision and efficiency compared to linear D-scoring tests.

Conclusions:

• The D-scoring method is well-suited for adaptive testing designs like MST due to its inherent item-level difficulty information.
• The developed MST framework provides a viable approach for simulating and evaluating D-scoring based adaptive tests.
• MST with D-scoring offers significant advantages in measurement precision and efficiency, outperforming traditional linear tests.