Item Selection With Collaborative Filtering in On-The-Fly Multistage Adaptive Testing.

Area of Science:

• Educational Measurement
• Computerized Adaptive Testing
• Psychometrics

Background:

• Item selection is crucial for adaptive testing accuracy and efficiency.
• Traditional methods like maximum Fisher information have limitations in optimizing item exposure and bank utilization.
• On-the-fly assembled multistage adaptive testing requires advanced item selection strategies.

Purpose of the Study:

• To introduce and evaluate collaborative filtering (CF) methods for item selection in on-the-fly assembled multistage adaptive testing.
• To compare the performance of user-based CF (UBCF) and item-based CF (IBCF) against the maximum Fisher information method.
• To assess the impact of CF methods on ability estimation accuracy, item exposure, and item bank utilization.

Main Methods:

• Simulated data under various test conditions (item bank size, test length, data sparsity).
• Implementation of user-based collaborative filtering (UBCF) and item-based collaborative filtering (IBCF) for item selection.
• Comparison with the maximum Fisher information method using metrics like measurement accuracy, item exposure rates, and item bank utilization.

Main Results:

• The UBCF method demonstrated superior performance in measurement accuracy compared to traditional methods.
• The IBCF method exhibited the most effective item bank utilization.
• Both CF methods showed potential for improving adaptive testing efficiency and effectiveness.

Conclusions:

• Collaborative filtering offers a promising alternative for item selection in multistage adaptive testing.
• UBCF and IBCF provide distinct advantages in accuracy and resource management, respectively.
• Further research is needed to explore the full potential and limitations of CF in adaptive testing frameworks.