Imputation Methods to Deal With Missing Responses in Computerized Adaptive Multistage Testing

Area of Science:

• Educational measurement
• Psychometrics
• Computerized adaptive testing

Background:

• Accurate examinee ability estimation is vital for effective routing in computerized adaptive multistage testing (CAMT).
• Missing responses can significantly complicate ability estimation, potentially leading to examinee misrouting.
• Careful consideration of missing data handling methods is therefore essential in CAMT.

Purpose of the Study:

• To investigate the performance of various missing data handling methods in CAMT.
• To compare imputation techniques, full information maximum likelihood (FIML), and scoring missing data as incorrect.
• To evaluate these methods under different missing data frameworks (MCAR, MAR, MNAR) and testing conditions.

Main Methods:

• Comparison of four missing data handling strategies: two imputation techniques, FIML, and scoring missing as incorrect.
• Evaluation under missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR) conditions.
• Benchmarking against a no-missing-data baseline.

Main Results:

• Imputation and FIML methods demonstrated superior performance compared to scoring missing data as incorrect.
• These preferred methods showed lower average bias and average root mean square error.
• Higher correlations between estimated and true examinee ability were observed with imputation and FIML.

Conclusions:

• Imputation and FIML are recommended for handling missing data in CAMT to ensure accurate ability estimation and routing.
• Scoring missing responses as incorrect can lead to significant estimation errors.
• The choice of missing data method impacts the reliability and validity of CAMT outcomes.