Evaluating the Performances of Missing Data Handling Methods in Ability Estimation From Sparse Data.

Area of Science:

• Educational Measurement and Psychometrics
• Statistical Modeling
• Data Analysis

Background:

• Missing data in educational assessments can significantly bias ability and item parameter estimations.
• Accurate parameter estimation requires appropriate methods for handling missing responses before analysis.

Purpose of the Study:

• To compare the performance of four distinct missing data handling methods for ability parameter estimation.
• To evaluate the accuracy of ability estimates under various missing data conditions using Monte Carlo simulations.

Main Methods:

• Conducted two Monte Carlo simulation studies to assess four missing data handling techniques: Full-Information Maximum Likelihood (FIML), zero replacement, Multiple Imputation with Chain Equations utilizing Classification and Regression Trees (MICE-CART), and Random Forest Imputation (MICE-RFI).
• Treated missing responses as a valid category for imputation methods to improve accuracy.
• Evaluated methods based on bias, root mean square error, and correlation between true and estimated ability parameters.

Main Results:

• Full-Information Maximum Likelihood (FIML) demonstrated superior performance across most simulated conditions.
• Zero replacement provided accurate estimates only when missing data proportions were extremely high.
• MICE-CART and MICE-RFI showed similar performance but were differentially affected by missing data mechanisms; their performance improved with more items and less missing data.

Conclusions:

• FIML is recommended as the most robust method for handling missing data in ability estimation.
• Imputation methods (MICE-CART, MICE-RFI) can be enhanced by incorporating missing data information, particularly in sparse datasets with missing at random mechanisms.
• The effectiveness of all methods improves with larger item sets and reduced missing data proportions.