A new item response theory model to adjust data allowing examinee choice

Area of Science:

• Psychometrics
• Network Analysis
• Statistical Modeling

Background:

• Traditional questionnaire testing restricts item selection due to challenges in estimating examinee ability and item difficulty.
• Existing item response theory (IRT) models face limitations when examinees can choose items, impacting statistical accuracy.

Purpose of the Study:

• To develop and evaluate a novel IRT model integrating network analysis for questionnaire data.
• To assess the model's performance in scenarios where examinees select a subset of items, particularly when standard model assumptions are violated.

Main Methods:

• Simulated three scenarios with varying degrees of assumption violation for the standard Rasch model.
• Introduced a new IRT model incorporating network analysis of questionnaire data.
• Compared the proposed model's item parameter recovery against the standard Rasch model.

Main Results:

• The novel IRT model demonstrated substantial improvements in item parameter recovery compared to the standard model.
• Accuracy of the new model was consistently closer to the reference parameter estimates across evaluated scenarios.
• The model successfully provided satisfactory IRT statistical estimates even when standard model assumptions were significantly violated.

Conclusions:

• Network analysis integrated into IRT offers a powerful approach for analyzing questionnaire data where item selection occurs.
• This new model advances psychometric methods, enabling more accurate ability and item difficulty estimation in complex testing situations.
• The findings represent a significant step towards overcoming technical limitations in current IRT applications for adaptive or choice-based testing.