Item Selection Methods for Computer Adaptive Testing With Passages.

Area of Science:

• Psychometrics
• Educational Measurement
• Computerized Adaptive Testing

Background:

• Computer adaptive testing (CAT) offers efficiency by shortening tests and enhancing latent trait estimation precision.
• Test design often involves multiple items related to a single passage, introducing dependencies.
• Accurate latent trait estimation is crucial for effective educational and psychological assessments.

Purpose of the Study:

• To investigate three computer adaptive testing (CAT) item selection techniques for items within the same passage.
• To compare the performance of the testlet-effect model, passage model, and unidimensional IRT model in CAT.
• To offer guidance on item selection methods for improved latent trait estimation in CAT.

Main Methods:

• A simulation study was conducted to compare three CAT item selection models: testlet-effect (T), passage (P), and unidimensional IRT (U).
• The testlet-effect model utilized a bifactor or constrained multidimensional IRT approach.
• Each model was evaluated under three item exposure control conditions: none, rate 0.2, and rate 1.

Main Results:

• The testlet-effect model demonstrated superior performance compared to the passage and unidimensional models.
• The passage and unidimensional models tended to overestimate the precision of latent trait estimates (theta).
• Item exposure control did not fundamentally alter the relative performance of the models.

Conclusions:

• The testlet-effect model is recommended for CAT when items are linked to passages, as it provides more accurate latent trait estimates.
• The passage and unidimensional models may lead to misleading conclusions about measurement precision.
• Careful selection of item-modeling techniques is essential for robust CAT development.