Adjacent-Categories Mokken Models for Rater-Mediated Assessments

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Mokken scale analysis is a probabilistic-nonparametric approach to scaling.
• Polytomous data, common in educational assessments, requires specialized scaling models.
• Existing polytomous Mokken models, based on cumulative probabilities, align with cumulative item response theory models.

Purpose of the Study:

• To present an adjacent-categories variation of polytomous Mokken models.
• To enhance conceptual alignment with educational performance assessments.
• To evaluate the diagnostic utility of the new model compared to original formulations.

Main Methods:

• Developed adjacent-categories variations of polytomous monotone homogeneity and double monotonicity models.
• Applied these models to data from a large-scale rater-mediated writing assessment.
• Compared results with the original polytomous Mokken model formulations.

Main Results:

• The adjacent-categories models provide additional diagnostic information on rater category usage.
• This diagnostic information is not available with the original polytomous Mokken models.
• The study illustrates the practical application of the adjacent-categories approach.

Conclusions:

• Adjacent-categories models offer a more nuanced understanding of rating processes in educational assessments.
• These models can improve methods for evaluating rater quality and consistency.
• The findings support the utility of adjacent-categories models for complex measurement scenarios.