Related Experiment Video
Updated: Dec 31, 2025
05:21
Computerized Adaptive Testing System of Functional Assessment of Stroke
Published on: January 7, 2019
6.2K
Adapting cognitive diagnosis computerized adaptive testing item selection rules to traditional item response theory
Miguel A Sorrel1, Juan R Barrada2, Jimmy de la Torre3
1Department of Social Psychology and Methodology, Universidad Autónoma de Madrid, Spain.
Plos One
|January 11, 2020
Summary
New item selection rules from cognitive diagnosis computerized adaptive testing (CD-CAT) can enhance traditional computerized adaptive testing (CAT) accuracy. The double Kullback-Leibler information (DKL) rule shows promise, especially in low-stakes testing scenarios.
Area of Science:
- Psychometrics
- Educational Measurement
- Computerized Adaptive Testing
Background:
- Traditional computerized adaptive testing (CAT) relies on item response theory.
- Cognitive diagnosis computerized adaptive testing (CD-CAT) utilizes cognitive diagnosis models.
- Item selection rules (ISRs) are crucial for CAT efficiency and accuracy.
Purpose of the Study:
- To evaluate the performance of CD-CAT ISRs (double Kullback-Leibler information - DKL, and generalized deterministic inputs, noisy "and" gate model discrimination index - GDI) within the traditional CAT framework.
- To compare the accuracy and test security of DKL and GDI against traditional ISRs (point Fisher information, weighted KL).
- To investigate the impact of different trait level estimation methods on ISR performance.
Main Methods:
- A simulation study was conducted to compare the performance of four ISRs: DKL, GDI, point Fisher information, and weighted KL.
- The study assessed accuracy, test security, and item overlap rates.
- The influence of trait level estimation methods (expected a posteriori vs. maximum likelihood) was examined.
Main Results:
- The DKL and GDI ISRs, particularly DKL, demonstrated potential to improve CAT accuracy.
- Improved accuracy with DKL came at the cost of higher item overlap.
- The choice of ISR became less impactful as the test length increased.
- DKL selected items with high 'a' parameters, while GDI selected items with low 'c' parameters.
- Expected a posteriori estimation was superior in early testing stages, converging with maximum likelihood later.
Conclusions:
- CD-CAT ISRs, especially DKL, can enhance traditional CAT accuracy, making them suitable for low-stakes testing where security is less critical.
- The selection of ISRs and trait level estimation methods impacts CAT performance, particularly in the initial stages of testing.
- Further research may explore the optimal application of these ISRs in various testing contexts.
