Related Experiment Video
Updated: Dec 12, 2025

Modeling the Size Spectrum for Macroinvertebrates and Fishes in Stream Ecosystems
Published on: July 30, 2019
On the Practical Consequences of Misfit in Mokken Scaling
Daniela Ramona Crişan1, Jorge N Tendeiro1, Rob R Meijer1
1University of Groningen, The Netherlands.
Abstract:
Mokken scale analysis is a popular method to evaluate the psychometric quality of clinical and personality questionnaires and their individual items. Although many empirical papers report on the extent to which sets of items form Mokken scales, there is less attention for the effect of violations of commonly used rules of thumb. In this study, the authors investigated the practical consequences of retaining or removing items with psychometric properties that do not comply with these rules of thumb. Using simulated data, they concluded that items with low scalability had some influence on the reliability of test scores, person ordering and selection, and criterion-related validity estimates. Removing the misfitting items from the scale had, in general, a small effect on the outcomes. Although important outcome variables were fairly robust against scale violations in some conditions, authors conclude that researchers should not rely exclusively on algorithms allowing automatic selection of items. In particular, content validity must be taken into account to build sensible psychometric instruments.
Insights
Mokken scale analysis is key for questionnaire quality. This study shows that while item issues can affect results, removing them often has minor impacts, emphasizing content validity alongside statistical rules.
Area of Science:
- Psychometrics
- Psychological Measurement
- Scale Analysis
Background:
- Mokken scale analysis is widely used for evaluating questionnaire psychometric properties.
- Empirical studies often focus on scale formation, with less attention to the impact of violating common rules of thumb.
Purpose of the Study:
- To investigate the practical consequences of retaining or removing items that do not meet Mokken scale analysis rules of thumb.
- To assess the influence of low scalability items on various psychometric outcomes.
Main Methods:
- Utilized simulated data to model the effects of item retention/removal.
- Examined the impact on test score reliability, person ordering and selection, and criterion-related validity.
Main Results:
- Items with low scalability demonstrated a discernible influence on reliability, person ordering, selection, and validity estimates.
- Removing misfitting items generally resulted in minor changes to the overall outcomes.
Conclusions:
- While some outcome variables showed robustness against scale violations, researchers should not solely depend on automated item selection algorithms.
- Content validity is crucial for constructing meaningful psychometric instruments, complementing statistical analyses.
Related Concept Videos
Scaling
Modeling and Similitude
Design Example: Creating a Hydraulic Model of a Dam Spillway
Mismatch Repair
The Mutator Protein Family Plays a Key Role in DNA Mismatch Repair
The human genome has more than 3 billion base pairs of DNA per cell. Prior to cell division, that vast amount of genetic...
Typical Model Studies
Transformers with Off-Nominal Turns Ratios

