Latent Poisson count models for action count data from technology-enhanced assessments.

Area of Science:

• Psychometrics and Educational Measurement
• Statistical Modeling
• Computational Statistics

Background:

• Computerized assessments increasingly use innovative item formats like drag-and-drop and scenario-based questions.
• Existing statistical models may not adequately capture the systematic influence of these diverse item types on user action counts.
• A flexible modeling approach is needed to account for item features and their impact on response data.

Purpose of the Study:

• To present a refinement scheme for statistical models that explicitly accounts for item features and item-type effects in computerized assessments.
• To integrate this scheme into existing count models, specifically the Rasch and Conway-Maxwell-Poisson models.
• To validate the proposed inferential procedures for accuracy and practical relevance.

Main Methods:

• Developed a refinement scheme using multifaceted parameterization to characterize common and unique item features.
• Implemented the scheme within Rasch and Conway-Maxwell-Poisson count models.
• Utilized Stan for model inference and conducted numerical experiments with simulated data for validation.

Main Results:

• The proposed inferential scheme demonstrated adequate recovery of underlying model parameters in simulations.
• Empirical application confirmed the practical relevance of the refinement when distinct item-type effects are present in the data.
• The study validated the estimation accuracy of the refined models.

Conclusions:

• The proposed model refinement effectively captures item-type effects in computerized assessment data.
• This approach offers a practical solution for analyzing count data from innovative item formats.
• Findings provide insights into applying Poisson models for count data analysis in psychometrics.