Flexible Computerized Adaptive Tests to Detect Misconceptions and Estimate Ability Simultaneously.

Area of Science:

• Psychometrics
• Educational Measurement
• Cognitive Science

Background:

• The Scaling Individuals and Classifying Misconceptions (SICM) model estimates misconceptions and general ability using discrete and continuous latent variables.
• Estimating both latent variables efficiently in linear tests presents challenges due to the SICM model's complexity.

Purpose of the Study:

• To propose a flexible computerized adaptive test (FCAT) design for efficient estimation of misconception profiles and general ability.
• To investigate the effectiveness of different item selection methods and transition criteria within an FCAT framework.

Main Methods:

• Developed three distinct item selection methods for adaptive testing.
• Introduced two transition criteria to manage adaptive steps based on estimation needs.
• Conducted two simulation studies to evaluate the proposed methods and criteria.

Main Results:

• The combination of item selection methods and transition criteria significantly impacts the estimation accuracy of latent variables.
• Specific combinations demonstrated superior performance in estimating either misconception profiles or general ability.
• The study provides empirical evidence for optimizing FCAT designs.

Conclusions:

• FCAT offers a more efficient approach to administering tests based on the SICM model compared to linear tests.
• The choice of item selection method and transition criterion should be tailored to the specific estimation goals.
• This research offers practical guidance for developing more accurate and efficient adaptive testing instruments.