A comparison of stopping rules for computerized adaptive screening measures using the rating scale model

Area of Science:

• Educational Measurement
• Psychometrics
• Computerized Adaptive Testing (CAT)

Background:

• Computerized adaptive testing (CAT) uses adaptive algorithms to tailor tests to individual ability levels.
• Selecting appropriate stopping rules is crucial for balancing test efficiency and measurement precision in CAT.
• Identifying at-risk students requires accurate and efficient assessment methods.

Purpose of the Study:

• To evaluate and compare three stopping rules for CAT: predicted standard error reduction (PSER), fixed-length, and minimum SE.
• To assess the effectiveness of these rules in classifying at-risk students using Andrich's rating scale model.
• To examine the impact of variables like trait distribution and item pool size on classification accuracy.

Main Methods:

• Simulated data were generated using Andrich's rating scale model.
• Three stopping rules (PSER, fixed-length, minimum SE) were applied to the simulated CAT data.
• Classification accuracy for at-risk students was evaluated under various conditions (trait distribution, item pool size).

Main Results:

• The PSER stopping rule demonstrated superior performance in correctly classifying at-risk students.
• PSER was effective in reducing the number of items administered, thereby alleviating test burden.
• The benefits of PSER were particularly evident with smaller item pools and specific trait distributions.

Conclusions:

• The PSER stopping rule is recommended for CAT applications focused on identifying at-risk students.
• PSER offers a balance between accurate classification and efficient testing, minimizing student burden.
• This finding is especially relevant for screening measures utilizing rating scale models with limited item sets.