Use of Crossed Random-Effects Models to Assess Multiple-Choice Items: An Experimental Study
Susana Sanz1,2, Carmen García2, Ricardo Olmos2
1Department of Psychology, https://ror.org/017mdc710Universidad Pontificia Comillas, Spain.
None:
Including general options on items, such as None-of-the-above (NOTA), could lead to worse psychometric properties. Further, personality traits are related to academic performance and could influence NOTA effects. Therefore, this study aims to test the effects of NOTA by manipulating its appearance and its use as the correct option or as a distractor, while considering the examinees' personality traits. The sample consisted of 449 psychology students who answered a statistics concept inventory. A crossed random-effects model was conducted to model the probability of answering an item correctly depending on items and students' covariates. The appearance of NOTA negatively affects the probability of answering correctly and changes between examinees. Conscientiousness, Extraversion, Emotional Stability, and previous knowledge of statistics are also significant predictors. The results point to increased items' difficulty when NOTA is included, although not all the examinees seem to show the same behavior when dealing with NOTA options.
Related Concept Videos
Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs
Crossover Experiments
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
Group Design
Randomized Experiments
Simple randomization
Simple...
Friedman Two-way Analysis of Variance by Ranks
Multiple Comparison Tests
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...


