A two-step item bank calibration strategy based on 1-bit matrix completion for small-scale computerized adaptive...

Area of Science:

• Educational Measurement
• Psychometrics
• Computer Science

Background:

• Computerized adaptive testing (CAT) offers personalized educational assessments but faces challenges in small-scale applications due to sparse data for item bank calibration.
• Existing methods struggle with limited sample sizes and incomplete response data, hindering CAT's broader adoption.

Purpose of the Study:

• To develop and evaluate a novel two-step item bank calibration strategy for small-scale assessments.
• To address the complexities of item calibration with sparse response data and small sample sizes.

Main Methods:

• A two-step item bank calibration strategy was developed, integrating the 1-bit matrix completion method with two incomplete pretesting designs.
• Two novel 1-bit matrix completion-based imputation methods were introduced to handle sparse data and limited sample sizes.
• Comparative assessment through simulation studies using varied pretesting designs, item bank structures, and sample sizes.

Main Results:

• The proposed 1-bit matrix completion methods demonstrated effectiveness in item calibration under sparse data conditions.
• Simulations showed competitive or superior performance compared to established methods for handling missing data in item parameter estimation.
• Practical application using empirical data from small-scale assessments validated the developed methods.

Conclusions:

• The novel 1-bit matrix completion-based imputation methods offer a viable solution for item bank calibration in small-scale CAT.
• These approaches enhance the feasibility and accuracy of personalized educational assessments in data-limited environments.
• The study provides practical tools for improving the application of CAT in diverse assessment contexts.